检索结果-内蒙古大学图书馆

Teaching parallel and distributed computing using data-intensive computing modules

JOURNAL OF PARALLEL AND DISTRIBUTED computing 2025年 202卷

作者： Gowanlock, Michael No Arizona Univ Sch Informat Comp & Cyber Syst Flagstaff AZ 86011 USA

Parallel and distributed computing (PDC) courses are useful for computer science (CS) and domain science students. For CS students, PDC is a fundamental field that examines concepts relating to a range of CS subfields, such as algorithms, architecture, simulation, software, systems, among others. Students with domain science backgrounds also require PDC to carry out their research objectives, and the ongoing data revolution has exacerbated this necessity. Given the rise of data science and other data-enabled computational fields, we propose several data-intensive pedagogic modules that are used to teach PDC using message-passing programming with the Message Passing Interface (MPI). These modules employ activities that are interesting, relevant, and accessible to both computer and domain science students enrolled in graduate level programs. Using pre-and post-module completion quizzes and anonymous free response surveys, we evaluated the efficacy of the pedagogic modules across four cohorts of students enrolled in a graduate level High Performance computing (HPC) course at Northern Arizona University. The students have diverse educational backgrounds as some students were enrolled in programs outside of CS. These programs include electrical and computer engineering, mechanical engineering, astronomy & planetary science, bioinformatics, and ecoinformatics. Despite the multidisciplinary backgrounds of the students, we find that the hands-on application-driven approach to teaching PDC was successful at helping students learn core PDC concepts, and that the modules are useful for facilitating online learning which was required during the COVID-19 pandemic.

关键词： Computer science education data-intensive computing High performance computing Parallel and distributed computing

来源：评论

学校读者我要写书评

暂无评论

DECO: Joint Computation Scheduling, Caching, and Communication in data-intensive computing Networks

引用

IEEE-ACM TRANSACTIONS ON NETWORKING 2022年第3期30卷 1058-1072页

作者： Kamran, Khashayar Yeh, Edmund Ma, Qian Pinterest Inc San Francisco CA 94107 USA Northeastern Univ Dept Elect & Comp Engn Boston MA 02115 USA Sun Yat Sen Univ Sch Intelligent Syst Engn Guangzhou 510275 Peoples R China

Driven by technologies such as IoT-enabled health care, machine learning applications at the edge, and industrial automation, mobile edge and fog computing paradigms have reinforced a general trend toward decentralized computing, where any network node can route traffic, compute tasks, and store data, possibly at the same time. In many such computing environments, there is a need to cache significant amounts of data, which may include large data sets, machine learning models, or executable code. In this work, we propose a framework for joint computation scheduling, caching, and request forwarding within such decentralized computing environments. We first characterize the stability region of a ``genie-aided'' computing network where data required by computation are instantly accessible, and develop a throughput optimal control policy for this model. Based on this, we develop a practically implementable distributed and adaptive algorithm, and show that it exhibits superior performance in terms of average task completion time, when compared to several baseline policies.

关键词： Task analysis Processor scheduling Throughput Computational modeling Edge computing Network topology Low latency communication Edge computing fog computing computation scheduling distributed computing data-intensive computing caching

来源：评论

学校读者我要写书评

暂无评论

data-intensive computing Modules for Teaching Parallel and Distributed computing

Data-Intensive Computing Modules for Teaching Parallel and D...

引用

35th IEEE International Parallel and Distributed Processing Symposium (IPDPS)

作者： Gowanlock, Michael Gallet, Benoit No Arizona Univ Sch Informat Comp & Cyber Syst Flagstaff AZ 86011 USA

ISBN: (纸本)9781665435772

Parallel and distributed computing (PDC) has found a broad audience that exceeds the traditional fields of computer science. This is largely due to the increasing computational demands of many engineering and domain science research objectives. Thus, there is a demonstrated need to train students with and without computer science backgrounds in core PDC concepts. Given the rise of data science and other data-enabled computational fields, we propose several data-intensive pedagogic modules that are used to teach PDC using message-passing programming with the Message Passing Interface (MPI). These modules employ activities that are common in database systems and scientific workflows that are likely to be employed by domain scientists. Our hypothesis is that using application-driven pedagogic materials facilitates student learning by providing the context needed to fully appreciate the goals of the activities. We evaluated the efficacy of using the data-intensive pedagogic modules to teach core PDC concepts using a sample of graduate students enrolled in a high performance computing course at Northern Arizona University. In the sample, only 30% of students have a traditional computer science background. We found that the hands-on application-driven approach was generally successful at helping students learn core PDC concepts.

关键词： Computer Science Education data-intensive computing High Performance computing Parallel and Distributed computing

来源：评论

学校读者我要写书评

暂无评论

Improvement of job completion time in data-intensive cloud computing applications

引用

JOURNAL OF CLOUD computing-ADVANCES SYSTEMS AND APPLICATIONS 2020年第1期9卷 1-20页

作者： Ibrahim, Ibrahim Adel Bassiouni, Mostafa Univ Cent Florida Dept Comp Engn Orlando FL 32816 USA Univ Technol Baghdad Dept Comp Engn Baghdad Iraq Univ Cent Florida Dept Comp Sci Orlando FL 32816 USA

Task stragglers in MapReduce jobs dramatically impede job execution of data-intensive computing in cloud data centers. This impedance is due to the uneven distribution of input data, heterogeneous data nodes, resource contention situations, and network configurations. data skew of intermediate data in MapReduce job causes delay failures due to the violation of job completion time. data-intensive computing frameworks, such as MapReduce or Hadoop YARN, employ HashPartitioner. This partitioner may cause intermediate data skew, which results in straggler reducers. In this paper, we strive to make Hadoop YARN more efficient in cloud environments. We present, a new partitioning scheme, called balanced data clusters partitioner (BDCP), to handle straggler Reduce tasks based on sampling of input data and feedback information about the current processing task. Our extensive experimental results show that BDCP can outperform the default Hadoop HashPartitioner and Range partitioner. BDCP can assist in straggler mitigation during reduce phase and minimize the job completion time in MapReduce jobs within data-intensive cloud computing.

关键词： Cloud computing MapReduce data-intensive computing Parallel and distributed processing Straggler reduce task Sampling

来源：评论

学校读者我要写书评

暂无评论

A Survey of Semantics-Aware Performance Optimization for data-intensive computing 15

A Survey of Semantics-Aware Performance Optimization for Dat...

引用

15th Intl Conf on Dependable, Autonomic and Secure computing, 15th Intl Conf on Pervasive Intelligence and computing, 3rd Intl Conf on Big data Intelligence and computing and Cyber Science and Technology Congress(DASC/PiCom/dataCom/CyberSciTech)

作者： Rao, Bingbing Wang, Liqang Univ Cent Florida Dept Comp Sci Orlando FL 32816 USA

ISBN: (纸本)9781538619568

We are living in the era of Big data and witnessing the explosion of data. Given that the limitation of CPU and I/O in a single computer, the mainstream approach to scalability is to distribute computations among a large number of processing nodes in a cluster or cloud. This paradigm gives rise to the term of data-intensive computing, which denotes a data parallel approach to process massive volume of data. Through the efforts of different disciplines, several promising programming models and a few platforms have been proposed for data-intensive computing, such as MapReduce, Hadoop, Apache Spark and Dyrad. Even though a large body of research work has being proposed to improve overall performance of these platforms, there is still a gap between the actual performance demand and the capability of current commodity systems. This paper is aimed to provide a comprehensive understanding about current semantics-aware approaches to improve the performance of data-intensive computing. We first introduce common characteristics and paradigm shifts in the evolution of data-intensive computing, as well as contemporary programming models and technologies. We then propose four kinds of performance defects and survey the state-of-the-art semantics-aware techniques. Finally, we discuss the research challenges and opportunities in the field of semantics-aware performance optimization for data-intensive computing.

关键词： data-intensive computing Big data Performance Optimization Semantics-Aware Compiler-based MapReduce Spark Hadoop

来源：评论

学校读者我要写书评

暂无评论

Performance Evaluation of data-intensive computing Applications on a Public IaaS Cloud

引用

COMPUTER JOURNAL 2016年第3期59卷 287-307页

作者： Exposito, Roberto R. Taboada, Guillermo L. Ramos, Sabela Tourino, Juan Doallo, Ramon Univ A Coruna Dept Elect & Syst Comp Architecture Grp Campus Elvina S-N La Coruna 15071 Spain

The advent of cloud computing technologies, which dynamically provide on-demand access to computational resources over the Internet, is offering new possibilities to many scientists and researchers. Nowadays, Infrastructure as a Service (IaaS) cloud providers can offset the increasing processing requirements of data-intensive computing applications, becoming an emerging alternative to traditional servers and clusters. In this paper, a comprehensive study of the leading public IaaS cloud platform, Amazon EC2, has been conducted in order to assess its suitability for data-intensive computing. One of the key contributions of this work is the analysis of the storage-optimized family of EC2 instances. Furthermore, this study presents a detailed analysis of both performance and cost metrics. More specifically, multiple experiments have been carried out to analyze the full I/O software stack, ranging from the low-level storage devices and cluster file systems up to real-world applications using representative data-intensive parallel codes and MapReduce-based workloads. The analysis of the experimental results has shown that data-intensive applications can benefit from tailored EC2-based virtual clusters, enabling users to obtain the highest performance and cost-effectiveness in the cloud.

关键词： data-intensive computing cloud computing Infrastructure as a Service Amazon EC2 cluster file system MapReduce

来源：评论

学校读者我要写书评

暂无评论

dispel4py: A Python framework for data-intensive scientific computing

引用

INTERNATIONAL JOURNAL OF HIGH PERFORMANCE computing APPLICATIONS 2017年第4期31卷 316-334页

作者： Filguiera, Rosa Krause, Amrey Atkinson, Malcolm Klampanos, Iraklis Moreno, Alexander Univ Edinburgh Sch Informat eSci Edinburgh Midlothian Scotland Univ Edinburgh EPCC Edinburgh Midlothian Scotland Natl Ctr Sci Res Demokritos Athens Greece Georgia Inst Technol Sch Comp Sci Atlanta GA 30332 USA

This paper presents dispel4py, a new Python framework for describing abstract stream-based workflows for distributed data-intensive applications. These combine the familiarity of Python programming with the scalability of workflows. data streaming is used to gain performance, rapid prototyping and applicability to live observations. dispel4py enables scientists to focus on their scientific goals, avoiding distracting details and retaining flexibility over the computing infrastructure they use. The implementation, therefore, has to map dispel4py abstract workflows optimally onto target platforms chosen dynamically. We present four dispel4py mappings: Apache Storm, message-passing interface (MPI), multi-threading and sequential, showing two major benefits: a) smooth transitions from local development on a laptop to scalable execution for production work, and b) scalable enactment on significantly different distributed computing infrastructures. Three application domains are reported and measurements on multiple infrastructures show the optimisations achieved;they have provided demanding real applications and helped us develop effective training. The *** is an open-source project to which we invite participation. The effective mapping of dispel4py onto multiple target infrastructures demonstrates exploitation of data-intensive and high-performance computing (HPC) architectures and consistent scalability.

关键词： data-intensive computing e-infrastructures data streaming scientific workflows programming frameworks

来源：评论

学校读者我要写书评

暂无评论

Improving Load Balance for data-intensive computing on Cloud Platforms

Improving Load Balance for Data-Intensive Computing on Cloud...

引用

IEEE International Conference on Smart Cloud (IEEE SmartCloud)

作者： Dai, Wei Ibrahim, Ibrahim Bassiouni, Mostafa Univ Cent Florida Dept Elect & Comp Engn Orlando FL 32816 USA Univ Cent Florida Dept Comp Sci Orlando FL 32816 USA

ISBN: (纸本)9781509052639

Nowadays, big data problems are ubiquitous, which in turn creates huge demand for data-intensive computing. The advent of Cloud computing has made data-intensive computing much more accessible and affordable than ever before. One of the crucial issues that can significantly affect the performance of data-intensive applications is the load balance among cluster nodes. In this paper, we address the load balance problem in the context of Hadoop Distributed File System (HDFS), a widely used file system for data-intensive computing on Cloud platforms, and present an innovative replica placement policy for HDFS, which can perfectly balance the computing load among all cluster nodes in both homogeneous and heterogeneous cluster environments.

关键词： Cloud computing data-intensive computing Load Balance Replica Placement Hadoop Distributed File System Hadoop MapReduce

来源：评论

学校读者我要写书评

暂无评论

data-intensive computing Infrastructure Systems for Unmodified Biological data Analysis Pipelines 11th

Data-Intensive Computing Infrastructure Systems for Unmodifi...

引用

11th International Meeting on Computational Intelligence Methods for Bioinformatics and Biostatistics (CIBB)

作者： Bongo, Lars Ailo Pedersen, Edvard Ernstsen, Martin Univ Tromso Dept Comp Sci Tromso Norway Univ Tromso Ctr Bioinformat Tromso Norway

ISBN: (纸本)9783319244624;9783319244617

Biological data analysis is typically implemented using a deep pipeline that combines a wide array of tools and databases. These pipelines must scale to very large datasets, and consequently require parallel and distributed computing. It is therefore important to choose a hardware platform and underlying data management and processing systems well suited for processing large datasets. There are many infrastructure systems for such data-intensive computing. However, in our experience, most biological data analysis pipelines do not leverage these systems. We give an overview of data-intensive computing infrastructure systems, and describe how we have leveraged these for: (i) scalable fault-tolerant computing for large-scale biological data;(ii) incremental updates to reduce the resource usage required to update large-scale compendium;and (iii) interactive data analysis and exploration. We provide lessons learned and describe problems we have encountered during development and deployment. We also provide a literature survey on the use of data-intensive computing systems for biological data processing. Our results show how unmodified biological data analysis tools can benefit from infrastructure systems for data-intensive computing.

关键词： data-intensive computing biological data analysis flexible pipelines infrastructure systems

来源：评论

学校读者我要写书评

暂无评论

In-Memory data Rearrangement for Irregular, data-intensive computing

引用

COMPUTER 2015年第8期48卷 18-25页

作者： Lloyd, Scott Gokhale, Maya Lawrence Livermore Natl Lab Livermore CA 94550 USA

The data rearrangement engine (DRE) performs in-memory data restructuring to accelerate irregular, data-intensive applications. An emulation on a field-programmable gate array shows how the DRE could improve speedup, ... 详细信息

关键词： field programmable gate arrays DRE data rearrangement engine energy consumption improvement field-programmable gate array in-memory data rearrangement in-memory data restructuring irregular-data-intensive application acceleration memory bandwidth improvement speedup improvement Computer applications Emulation Field programmable gate arrays Memory management Stacked memory 3D stacked memory FPGA emulation active memory data rearrangement engine data-intensive computing irregular applications processing in memory

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：