检索结果-内蒙古大学图书馆

distributed Fusion of Heterogeneous Remote Sensing and Social Media Data: A Review and New Developments

PROCEEDINGS OF THE IEEE 2021年第8期109卷 1350-1363页

作者： Li, Jun Liu, Zhenjie Lei, Xinya Wang, Lizhe Sun Yat Sen Univ Sch Geog & Planning Guangdong Prov Key Lab Urbanizat & Geosimulat Guangzhou 510275 Peoples R China China Univ Geosci Wuhan Sch Comp Sci Wuhan 430074 Peoples R China

Despite the wide availability of remote sensing big data from numerous different Earth Observation (EO) instruments, the limitations in the spatial and temporal resolution of such EO sensors (as well as atmospheric opacity and other kinds of interferers) have led to many situations in which using only remote sensing data cannot fully meet the requirements of applications in which a (near) real-time response is needed. Examples of these applications include floods, earthquakes, and other kinds of natural disasters, such as typhoons. To address this issue, social media data have gradually been adopted to fill possible gaps in the analysis when remote sensing data are lacking or incomplete. In this case, the fusion of heterogeneous big data streams from multiple data sources introduces significant demands from a computational viewpoint. In order to meet these challenges, distributed computing is increasingly viewed as a feasible solution to parallelize the analysis of massive data coming from different sources (e.g., remote sensing and social media data). In this article, we provide an overview of available and new distributed strategies to address the computational challenges brought by massive heterogeneous data processing and fusion for real-time environmental monitoring and decision-making. The 2013 Boulder (Colorado) flood event is taken as a case study to evaluate several new distributed data fusion frameworks. Experimental results demonstrate that the proposed distributed frameworks are suitable in terms of response time and computational requirements for fusing large-volume heterogeneous data sources.

关键词： Remote sensing distributed databases Earth Artificial satellites Monitoring MODIS Temperature sensors Data fusion parallel and distributed computing remote sensing social media

来源：评论

学校读者我要写书评

暂无评论

A survey of genome sequence assembly techniques and algorithms using high-performance computing

引用

JOURNAL OF SUPERcomputing 2015年第1期71卷 293-339页

作者： Ahmed, Munib Ahmad, Ishfaq Ahmad, Mohammad Saad Univ Texas Arlington Arlington TX 76019 USA

Genome assembly has been an area of active research since the DNA structure was discovered and has gathered more steam after the Human Genome project was launched. A large number of genomes have been assembled and many more are in the pipeline. A number of full-scale assemblers and other special-purpose modules have been reported. Since the volume of data involved in the genome assembly process is extraordinarily large and requires significantly large computational power and processing time, many assemblers have utilized parallel computing to achieve faster and more efficient reconstruction of the DNA. A genome assembler is a multi-step process including different components that may be partly or fully parallelized. Although several assemblers and individual modules that perform various tasks, such as pairwise alignment, multiple sequence alignment, and repeat finding, have been analyzed and documented before, this paper provides a holistic view of the assembly process in the realm of parallel and distributed computing, streamlining all the individual tasks related, but not limited to, the whole genome shotgun sequencing into a sequence of loosely coupled stages where one stage consumes the output of the preceding stage and passes its results to the next one. Many of these tasks are essential to the current and next-generation sequence assemblers. The paper walks through the entire streamlined process while describing, analyzing, and commenting on the algorithms and techniques that have been designed and implemented for each of the stages. Where applicable, the paper suggests improvements that may form the basis of potentially new research work.

关键词： parallel and distributed computing Whole genome sequence assembly Sequence alignment

来源：评论

学校读者我要写书评

暂无评论

parallel lossless data compression using the Burrows-Wheeler Transform

引用

INTERNATIONAL JOURNAL OF WEB AND GRID SERVICES 2008年第1期4卷 117-135页

作者： Gilchrist, Jeff Cuhadar, Aysegul Carleton Univ Dept Syst & Comp Engn Ottawa ON K1S 5B6 Canada

In this paper, we present parallel algorithms for lossless data compression based on the Burrows-Wheeler Transform (BWT) block-sorting technique. We investigate the performance of using data parallelism and task parallelism for both multi-threaded and message-passing programming. The output produced by the parallel algorithms is fully compatible with their sequential counterparts. To balance the workload among processors we develop a task scheduling strategy. An extensive set of experiments is performed with a shared memory NUMA system using up to 120 processors and on a distributed memory cluster using up to 100 processors. Our experimental results show that significant speedup can be achieved with both data parallel and task parallel methodologies. These algorithms will greatly reduce the amount of time it takes to compress large amounts of data while the compressed data remains in a form that users without access to multiple processor systems can still use.

关键词： lossless data compression Burrows-Wheeler Transform BWT parallel and distributed computing bzip2 Message Passing Interface MPI multi-threading

来源：评论

学校读者我要写书评

暂无评论

Evaluating high-performance computers

引用

CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE 2005年第10期17卷 1239-1270页

作者： Vetter, JS de Supinski, BR Kissel, L May, J Vaidya, S Oak Ridge Natl Lab Oak Ridge TN 37992 USA Lawrence Livermore Natl Lab Livermore CA 94551 USA

Comparisons of high-performance computers based on their peak floating point performance are common but seldom useful when comparing performance on real workloads. Factors that influence sustained performance extend beyond a system's floating-point units, and real applications exercise machines in complex and diverse ways. Even when it is possible to compare systems based on their performance, other considerations affect which machine is best for a given organization. These include the cost, the facilities requirements (power, floorspace, etc.), the programming model, the existing code base, and so on. This paper describes some of the important measures for evaluating high-performance computers. We present data for many of these metrics based on our experience at Lawrence Livermore National Laboratory (LLNL), and we compare them with published information on the Earth Simulator. We argue that evaluating systems involves far more than comparing benchmarks and acquisition costs. We show that evaluating systems often involves complex choices among a variety of factors that influence the value of a supercomputer to an organization, and that the high-end computing community should view cost/performance comparisons of different architectures with skepticism. Published in 2005 by John Wiley & Sons, Ltd.

关键词： high-performance computing performance evaluation computer architecture parallel and distributed computing

来源：评论

学校读者我要写书评

暂无评论

Alternative model representations and computing capacity: Implications for model management

引用

DECISION SUPPORT SYSTEMS 2006年第3期42卷 1413-1430页

作者： Gnanendran, Kingsley Sundarraj, R. P. Univ Waterloo Dept Management Sci Waterloo ON N2L 3G1 Canada Univ Scranton Dept Operat & Informat Mangement Scranton PA 18510 USA

Recent research on model management systems (MMS) recognizes the importance of considering potential algorithmic performance in the selection of an appropriate model to solve a real-world problem. Model selection, as typically viewed in the literature however, is the process of selecting from among alternative model classes, rather than from alternative mathematical representations of the same model class. In this paper, we take up this subtler aspect of model selection, and provide tangible evidence that shows how just changing the representation of a model can have a dramatic impact on algorithmic performance. Using problem decomposition and distributed processing, we conduct a series of computational experiments to study the interrelationships between model representation, computing capacity, and algorithmic performance. We discuss potential implications of our results for improving MMS design and address a key prerequisite for the enhanced design, by proposing and validating an approach for solution time prediction. (c) 2005 Elsevier B.V. All rights reserved.

关键词： model management systems distributed model management model selection parallel and distributed computing grid computing metacomputing decomposition

来源：评论

学校读者我要写书评

暂无评论

The state of the art and taxonomy of big data analytics: view from new big data framework

引用

ARTIFICIAL INTELLIGENCE REVIEW 2020年第2期53卷 989-1037页

作者： Mohamed, Azlinah Najafabadi, Maryam Khanian Wah, Yap Bee Zaman, Ezzatul Akmal Kamaru Maskat, Ruhaila Univ Teknol MARA Adv Analyt Engn Ctr Fac Comp & Math Sci Shah Alam Selangor Malaysia INTI Int Univ & Coll Fac Informat Technol Nilai Negeri Sembilan Malaysia

Big data has become a significant research area due to the birth of enormous data generated from various sources like social media, internet of things and multimedia applications. Big data has played critical role in many decision makings and forecasting domains such as recommendation systems, business analysis, healthcare, web display advertising, clinicians, transportation, fraud detection and tourism marketing. The rapid development of various big data tools such as Hadoop, Storm, Spark, Flink, Kafka and Pig in research and industrial communities has allowed the huge number of data to be distributed, communicated and processed. Big data applications use big data analytics techniques to efficiently analyze large amounts of data. However, choosing the suitable big data tools based on batch and stream data processing and analytics techniques for development a big data system are difficult due to the challenges in processing and applying big data. Practitioners and researchers who are developing big data systems have inadequate information about the current technology and requirement concerning the big data platform. Hence, the strengths and weaknesses of big data technologies and effective solutions for Big Data challenges are needed to be discussed. Hence, due to that, this paper presents a review of the literature that analyzes the use of big data tools and big data analytics techniques in areas like health and medical care, social networking and internet, government and public sector, natural resource management, economic and business sector. The goals of this paper are to (1) understand the trend of big data-related research and current frames of big data technologies;(2) identify trends in the use or research of big data tools based on batch and stream processing and big data analytics techniques;(3) assist and provide new researchers and practitioners to place new research activity in this domain appropriately. The findings of this study will provide insights

关键词： parallel and distributed computing Big data tools Big data analytics techniques Domain area

来源：评论

学校读者我要写书评

暂无评论

DS-ADMM plus plus : A Novel distributed Quantized ADMM to Speed up Differentially Private Matrix Factorization

引用

IEEE TRANSACTIONS ON parallel AND distributed SYSTEMS 2022年第6期33卷 1289-1302页

作者： Zhang, Feng Xue, Erkang Guo, Ruixin Qu, Guangzhi Zhao, Gansen Zomaya, Albert Y. China Univ Geosci Sch Comp Sci Wuhan 430074 Hubei Peoples R China Oakland Univ Dept Engn & Comp Sci Rochester MI 48309 USA South China Normal Univ Sch Comp Sci Guangzhou 510631 Guangdong Peoples R China Univ Sydney Sch Comp Sci Camperdown NSW 2006 Australia

Matrix factorization is a powerful method to implement collaborative filtering recommender systems. This article addresses two major challenges, privacy and efficiency, which matrix factorization is facing. We based our work on DS-ADMM, a distributed matrix factorization algorithm with decent efficiency, to achieve the following two pieces of work: (1) Integrated local differential privacy paradigm into DS-ADMM to provide the privacy-preserving property;(2) Introduced a stochastic quantized function to reduce transmission overheads in ADMM to further improve efficiency. We named our work DS-ADMM++, in which one '+' refers to differential privacy, and the other '+' refers to quantized techniques. DS-ADMM++ is the first to perform efficient and private matrix factorization under the scenarios of differential privacy and DS-ADMM. We conducted experiments with benchmark data sets to demonstrate that our approach provides differential privacy and excellent scalability with a decent loss of accuracy.

关键词： Differential privacy Quantization (signal) Matrix decomposition Recommender systems Charge coupled devices Machine learning Costs Matrix factorization parallel and distributed computing differential privacy ADMM quantization

来源：评论

学校读者我要写书评

暂无评论

Protein structure prediction using distributed parallel particle swarm optimization

引用

NATURAL computing 2013年第1期12卷 29-41页

作者： Kondov, Ivan Karlsruhe Inst Technol Steinbuch Ctr Comp D-76021 Karlsruhe Germany

Particle swarm optimization is a powerful technique for computer aided prediction of proteins' three-dimensional structure. In this work, employing an all-atom force field and the standard algorithm, as implemented in the ArFlock library in previous work, the low-energy conformations of several peptides of different sizes in vacuum starting from completely extended conformations are investigated. The computed structures are in good overall agreement with experimental data and results from other computer simulations. Periodic boundary conditions applied to the search space improve the performance of the method dramatically, especially when the linear velocity update rule is used. It is also shown that asynchronous parallelization speeds up the simulation better than the synchronous one and reduces the effective time for predictions significantly.

关键词： Protein structure prediction Particle swarm optimization Global optimization Force field parallel and distributed computing

来源：评论

学校读者我要写书评

暂无评论

Asynchronous optimisation with the use of a cascade search algorithm

引用

COMPUTERS & CHEMICAL ENGINEERING 2014年 66卷 276-289页

作者： Cecelja, F. Kokossis, A. Du, D. Yang, S. Univ Surrey Ctr Proc & Informat Syst Engn Guildford GU2 7XH Surrey England Natl Tech Univ Athens Sch Chem Engn GR-15780 Athens Greece

This paper introduces the development of an asynchronous approach coupled with a cascade optimisation algorithm. The approach incorporates concepts of asynchronous Markov processes and introduces a search process that is benefiting from distributed computing infrastructures. The algorithm uses concepts of partitions and pools to store intermediate solutions and corresponding objectives. Population inflections are performed periodically to ensure that Markov processes, still independent and asynchronous, make arbitrary use of intermediate solutions. Tested against complex optimisation problems and in comparison with commonly used Tabu Search, the asynchronous cascade algorithm demonstrates a significant potential in distributed operations with favourable comparisons drawn against synchronous and quasi-asynchronous versions of conventional algorithms. (c) 2014 Elsevier Ltd. All rights reserved.

关键词： Markov processes Asynchronous optimisation parallel and distributed computing

来源：评论

学校读者我要写书评

暂无评论

Layer-wise partitioning and merging for efficient and scalable deep learning

引用

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2023年第1期149卷 432-444页

作者： Akintoye, S. B. Han, L. Lloyd, H. Zhang, X. Dancey, D. Chen, H. Zhang, D. Manchester Metropolitan Univ Dept Comp & Math Manchester England Univ Sheffield Dept Comp Sci Sheffield England Nanjing Univ Aeronaut & Astronaut Coll Comp Sci & Technol Nanjing Peoples R China

Deep Neural Network (DNN) models are usually trained sequentially from one layer to another, which causes forward, backward and update locking problems, leading to poor performance in terms of training time. The existing parallel strategies to mitigate these problems provide suboptimal runtime performance. In this work, we have proposed a novel layer-wise partitioning and merging, forward and backward pass parallel framework to provide better training performance. The novelty of the proposed work consists of (1) a layer-wise partition and merging model which can minimise communication overhead between devices without the memory cost of existing strategies during the training process;(2) a forward pass and backward pass parallelisation to address the update locking problem and minimise the total training cost. The experimental evaluation on real use cases shows that the proposed method outperforms the state-of-the-art approaches in terms of training speed;and achieves almost linear speedup without compromising the accuracy performance of the non-parallel approach.& COPY;2023 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY license (http://***/licenses/by/4.0/).

关键词： Deep Neural Network Deep learning Data parallelism Model parallelism parallel and distributed computing High performance computing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：