检索结果-内蒙古大学图书馆

distributed Fusion of Heterogeneous Remote Sensing and Social Media Data: A Review and New Developments

PROCEEDINGS OF THE IEEE 2021年第8期109卷 1350-1363页

作者： Li, Jun Liu, Zhenjie Lei, Xinya Wang, Lizhe Sun Yat Sen Univ Sch Geog & Planning Guangdong Prov Key Lab Urbanizat & Geosimulat Guangzhou 510275 Peoples R China China Univ Geosci Wuhan Sch Comp Sci Wuhan 430074 Peoples R China

Despite the wide availability of remote sensing big data from numerous different Earth Observation (EO) instruments, the limitations in the spatial and temporal resolution of such EO sensors (as well as atmospheric opacity and other kinds of interferers) have led to many situations in which using only remote sensing data cannot fully meet the requirements of applications in which a (near) real-time response is needed. Examples of these applications include floods, earthquakes, and other kinds of natural disasters, such as typhoons. To address this issue, social media data have gradually been adopted to fill possible gaps in the analysis when remote sensing data are lacking or incomplete. In this case, the fusion of heterogeneous big data streams from multiple data sources introduces significant demands from a computational viewpoint. In order to meet these challenges, distributed computing is increasingly viewed as a feasible solution to parallelize the analysis of massive data coming from different sources (e.g., remote sensing and social media data). In this article, we provide an overview of available and new distributed strategies to address the computational challenges brought by massive heterogeneous data processing and fusion for real-time environmental monitoring and decision-making. The 2013 Boulder (Colorado) flood event is taken as a case study to evaluate several new distributed data fusion frameworks. Experimental results demonstrate that the proposed distributed frameworks are suitable in terms of response time and computational requirements for fusing large-volume heterogeneous data sources.

关键词： Remote sensing distributed databases Earth Artificial satellites Monitoring MODIS Temperature sensors Data fusion parallel and distributed computing remote sensing social media

来源：评论

学校读者我要写书评

暂无评论

Introducing parallel computing in a Second CS Course 36

Introducing Parallel Computing in a Second CS Course

引用

36th IEEE International parallel and distributed Processing Symposium (IEEE IPDPS)

作者： Newhall, Tia Webb, Kevin C. Chaganti, Vasanta Danner, Andrew Swarthmore Coll Comp Sci Dept Swarthmore PA 19081 USA

ISBN: (纸本)9781665497473

The ubiquity of multicore processors, cloud computing, and hardware accelerators have elevated parallel and distributed computing (PDC) topics into fundamental building blocks of the undergraduate CS curriculum. Therefore, it is increasingly important for students to learn a common core of introductory PDC topics and develop parallel thinking skills early in their CS studies. We present the curricular design, pedagogy, and goals of an introductory-level course on computer systems that introduces parallel computing to students who have only a CS1 background. Our course focuses on three curricular goals that serve to integrate the ACM-IEEE TCPP guidelines throughout: a vertical slice through the computer of how it runs a program;evaluating system costs associated with running a program;and taking advantage of the power of parallel computing. We elaborate on the goals and details of our course's key modules, and we discuss our pedagogical approach that includes active-learning techniques. We find that the PDC foundation gained through early exposure in this course helps students gain confidence in their ability to expand and apply their understanding of PDC concepts throughout their CS education.

关键词： parallel and distributed computing CS Education curriculum design

来源：评论

学校读者我要写书评

暂无评论

BaPa: A Novel Approach of Improving Load Balance in parallel Matrix Factorization for Recommender Systems

引用

IEEE TRANSACTIONS ON COMPUTERS 2021年第5期70卷 789-802页

作者： Guo, Ruixin Zhang, Feng Wang, Lizhe Zhang, Wusheng Lei, Xinya Ranjan, Rajiv Zomaya, Albert Y. China Univ Geosci Sch Comp Sci Wuhan 430074 Peoples R China Tsinghua Univ Dept Comp Sci Beijing 100084 Peoples R China Univ Newcastle Sch Comp Newcastle Upon Tyne NE1 7RU Tyne & Wear England Univ Sydney Sch Comp Sci Camperdown NSW 2006 Australia

A simplified approach to accelerate matrix factorization of big data is to parallelize it. A commonly used method is to divide the matrix into multiple non-intersecting blocks and concurrently calculate them. This operation causes the Load balance problem, which significantly impacts parallel performance and is a big concern. A general belief is that the load balance across blocks is impossible by balancing rows and columns separately. We challenge the belief by proposing an approach of "Balanced Partitioning (BaPa)". We demonstrate under what circumstance independently balancing rows and columns can lead to the balanced intersection of rows and columns, why, and how. We formally prove the feasibility of BaPa by observing the variance of rating numbers across blocks, and empirically validate its soundness by applying it to two standard parallel matrix factorization algorithms, DSGD and CCD++. Besides, we establish a mathematical model of "Imbalance Degree" to explain further why BaPa works well. BaPa is applied to synchronous parallel matrix factorization, but as a general load balance solution, it has significant application potential.

关键词： Matrix decomposition Task analysis Partitioning algorithms Charge coupled devices Sparse matrices Recommender systems Computer science Matrix factorization stochastic gradient descent recommender systems parallel and distributed computing load balancing

来源：评论

学校读者我要写书评

暂无评论

Near-optimal large-scale k-medoids clustering

引用

INFORMATION SCIENCES 2021年 545卷 344-362页

作者： Ushakov, Anton V. Vasilyev, Igor Russian Acad Sci Matrosov Inst Syst Dynam & Control Theory Siberian Branch 134 Lermontov Str Irkutsk 664033 Russia

The k-medoids (k-median) problem is one of the best known unsupervised clustering problems. Due to its complexity, finding high-quality solutions for huge-scale datasets remains extremely challenging. The application of many approaches finding optimal or quality solutions is limited to only small and medium-size instances. On the other hand, many parallel, distributed algorithms that can handle huge-scale datasets usually provide very poor solutions. In this paper, we develop a first parallel, distributed primal-dual heuristic algorithm for the k-medoids problem. Its main component is a very efficient parallel subgradient column generation that solves a Lagrangian dual problem and finds a tight bound on solution quality. High-quality solutions are then produced by a parallel core selection technique. We considerably reduce computational burden and memory load by employing a nearest neighbor strategy to approximate the dissimilarity matrix. We demonstrate that our algorithm finds very close to optimal solutions, confirmed by the tightness of dual bounds, of instances that are much larger than those considered in the literature to date. Our experiments include clustering large-scale collections of face images into several thousand of clusters. We show that our approach outperforms parallel improved versions of the most popular k-medoids clustering algorithms, achieving nearly linear parallel speedup. (C) 2020 Elsevier Inc. All rights reserved.

关键词： parallel and distributed computing K-medoids clustering Nearest neighbors P-median problem MPI CLARA

来源：评论

学校读者我要写书评

暂无评论

Effects of network topology on the performance of consensus and distributed learning of SVMs using ADMM

引用

PEERJ COMPUTER SCIENCE 2021年 7卷 e397页

作者： Tavara, Shirin Schliep, Alexander Chalmers Univ Technol Dept Comp Sci & Engn Data Sci & AI Div Gothenburg Sweden Univ Gothenburg Gothenburg Sweden

The Alternating Direction Method of Multipliers (ADMM) is a popular and promising distributed framework for solving large-scale machine learning problems. We consider decentralized consensus-based ADMM in which nodes may only communicate with one-hop neighbors. This may cause slow convergence. We investigate the impact of network topology on the performance of an ADMM-based learning of Support Vector Machine using expander, and mean-degree graphs, and additionally some of the common modern network topologies. In particular, we investigate to which degree the expansion property of the network influences the convergence in terms of iterations, training and communication time. We furthermore suggest which topology is preferable. Additionally, we provide an implementation that makes these theoretical advances easily available. The results show that the performance of decentralized ADMM-based learning of SVMs in terms of convergence is improved using graphs with large spectral gaps, higher and homogeneous degrees.

关键词： Machine learning parallel and distributed computing SVMs ADMM Expander graphs distributed optimization Convergence

来源：评论

学校读者我要写书评

暂无评论

Leveraging teaching on demand: Approaching HPC to undergrads

引用

JOURNAL OF parallel and distributed computing 2021年 156卷 148-162页

作者： Catalan, Sandra Carratala-Saez, Rocio Iserte, Sergio Univ Complutense Madrid UCM Madrid Spain Univ Jaume I UJI Castellon de La Plana Spain

High Performance computing (HPC) is a highly demanded discipline in companies and institutions. However, as students and also afterwards as professors, we observed a lack of HPC related content in the engineering degrees at our university, including Computer Science. Thus, we designed and offered the engineering students a non-mandatory course entitled "Build your owncluster employing Raspberry Pi" to provide the students with HPC skills. With this course, we covered the basics of supercomputing (hardware, networking, software tools, performance evaluation, cluster management, etc.). This was possible thanks to leveraging the flexibility and versatility of Raspberry Pi devices, and the students' motivation that arose from the hands-on experience. Moreover, the course included a "Teaching on demand" component to let the attendees choose a field to explore, based on their own interests. In this paper, we offer all the details to let anyone fully reproduce the course. Besides, we analyze and evaluate the methodology that let us fulfill our objectives: increase the students' HPC skills and knowledge in such a way that they feel capable of utilizing it in their mid-term professional career. (c) 2021 Elsevier Inc. All rights reserved.

关键词： Computational cluster Undergrad teaching System administration parallel and distributed computing Raspberry Pi

来源：评论

学校读者我要写书评

暂无评论

A scalable and flexible basket analysis system for big transaction data in Spark

引用

INFORMATION PROCESSING & MANAGEMENT 2024年第2期61卷

作者： Sun, Xudong Ngueilbaye, Alladoumbaye Luo, Kaijing Cai, Yongda Wu, Dingming Huang, Joshua Zhexue Shenzhen Univ Natl Engn Lab Big Data Syst Comp Technol Shenzhen 518060 Peoples R China Shenzhen Univ Big Data Inst Coll Comp Sci & Software Engn Shenzhen 518060 Peoples R China Guangdong Lab Artificial Intelligence & Digital Ec Shenzhen 518107 Peoples R China

Basket analysis is a prevailing technique to help retailers uncover patterns and associations of sold products in customer shopping transactions. However, as the size of transaction databases grows, the traditional basket analysis techniques and systems become less effective because of two issues in the applications of the big data age: data scalability and flexibility to adapt different application tasks. This paper proposes a scalable distributed frequent itemset mining (ScaDistFIM) algorithm for basket analysis on big transaction data to solve these two problems. ScaDistFIM is performed in two stages. The first stage uses the FP-Growth algorithm to compute the local frequent itemsets from each random subset of the distributed transaction dataset, and all random subsets are computed in parallel. The second stage uses an approximation method to aggregate all local frequent itemsets to the final approximate set of frequent itemsets where the support values of the frequent itemsets are estimated. We further elaborate on implementing the ScaDistFIM algorithm and a flexible basket analysis system using Spark SQL queries to demonstrate the system's flexibility in real applications. The experiment results on synthetic and real-world transaction datasets demonstrate that compared to the Spark FP-Growth algorithm, the ScaDistFIM algorithm can achieve time savings of at least 90% while ensuring nearly 100% accuracy. Hence, the ScaDistFIM algorithm exhibits superior scalability. On dataset GenD with 1 billion records, the ScaDistFIM algorithm requires only 360 s to achieve 100% precision and recall. In contrast, due to memory limitations, Spark FP-Growth cannot complete the computation task.

关键词： Big transaction data Frequent itemset mining parallel and distributed computing Business basket analysis Basket analysis systems

来源：评论

学校读者我要写书评

暂无评论

Unplugged parallelism for First-Year CS Majors 2022

Unplugged Parallelism for First-Year CS Majors

引用

53rd Annual ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE)

作者： Anthony, Barbara M. Erdil, D. Cenk Glebova, Olga Montante, Robert Southwestern Univ Georgetown 78626 TX Guyana Sacred Heart Univ Fairfield CT 06825 USA Georgia State Univ Atlanta GA 30302 USA Bloomsburg Univ Bloomsburg PA 17815 USA

ISBN: (纸本)9781450390705

We use "unplugged" activities to introduce parallel concepts in a first-year seminar for Computer Science majors. Student teams explore parallel approaches to computational tasks. Pre- and post-activity surveys, and a reflection paper, measure the impact of these activities on students' views about parallel programming. Our goal is to encourage parallel thinking about programming tasks before sequential approaches become ingrained. Computer Science curricula have traditionally focused on sequential approaches to programming, which were well matched to earlier computer systems. However, current systems almost all use multiprocessor CPUs, and are frequently used in clusters or networks of multiple computers. Recent curricular guidelines from organizations such as ACM and ABET recommend exposure to parallel computing concepts.

关键词： parallel and distributed computing unplugged activities computing education

来源：评论

学校读者我要写书评

暂无评论

A novel hybrid resampling algorithm for parallel/distributed particle filters

引用

JOURNAL OF parallel and distributed computing 2021年 151卷 24-37页

作者： Zhang, Xudong Zhao, Liang Zhong, Wei Gu, Feng CUNY Grad Ctr Dept Comp Sci New York NY USA CUNY Lehman Coll Dept Comp Sci Bronx NY USA Univ South Carolina Upstate Div Math & Comp Sci Spartanburg SC USA CUNY Coll Staten Isl Dept Comp Sci Staten Isl NY 10314 USA

parallel/distributed particle filters have been widely used in the estimation of states of dynamic systems by using multiple processing units (PUs). In parallel/distributed particle filters, the centralized resampling needs a central unit (CU) to serve as a hub to execute the global resampling. The centralized scheme is the main obstacle for the improved performance due to its global nature. To reduce the communication cost, the decentralized resampling was proposed, which only conducted the resampling on each PU. Although the decentralized resampling can improve the performance, it suffers from the low accuracy due to the local nature. Therefore, we propose a novel hybrid resampling algorithm to dynamically adjust the intervals between the centralized resampling steps and the decentralized resampling steps based on the measured system convergence. We formulate the proposed algorithm and prove it to be uniformly convergent. Since the proposed algorithm is a generalization of various versions of the hybrid resampling, its proof provides the solid theoretical foundation for their wide adoptions in parallel/distributed particle filters. In the experiments, we evaluate and compare different resampling algorithms including the centralized resampling algorithm, the decentralized resampling algorithm, and different types of existing hybrid resampling algorithms to show the effectiveness and the improved performance of the proposed hybrid resampling algorithm. (C) 2021 Elsevier Inc. All rights reserved.

关键词： Particle filters Sequential Monte Carlo methods Convergence parallel and distributed computing Resampling

来源：评论

学校读者我要写书评

暂无评论

A NEW CLASS OF AMG INTERPOLATION METHODS BASED ON MATRIX-MATRIX MULTIPLICATIONS

引用

SIAM JOURNAL ON SCIENTIFIC computing 2021年第5期43卷 S540-S564页

作者： LI, Ruipeng Sjogreen, Bjorn Yang, Ulrike Meier Ctr Appl Sci Comp Lawrence Livermore Natl Lab Livermore CA 94551 USA

A new class of distance-two interpolation methods for algebraic multigrid (AMG) that can be formulated in terms of sparse matrix-matrix multiplications is presented and analyzed. Compared with similar distance-two prolongation operators [H. De Sterck et al., Numer. Linear Algebra Appl., 15 (2008), pp. 115-139], the proposed algorithms exhibit improved efficiency and portability to various computing platforms, since they allow one to easily exploit existing high-performance sparse matrix kernels. The new interpolation methods have been implemented in hypre [R. D. Falgout and U. M. Yang, hypre: A library of high performance preconditioners, in Computational Science - ICCS 2002, P. M. A. Sloot et al., eds., Springer, Berlin, Heidelberg, 2002, pp. 632-641], a widely used parallel multigrid solver library. With the proposed interpolations, the overall time of hypre's BoomerAMG setup can be considerably reduced, while sustaining equivalent, sometimes improved, convergence rates. Numerical results for a variety of test problems on parallel machines are presented that support the superiority of the proposed interpolation operators over the existing ones in hypre.

关键词： AMG interpolation sparse matrix computation parallel and distributed computing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：