检索结果-内蒙古大学图书馆

Performance Evaluation of data-intensive computing Applications on a Public IaaS Cloud

COMPUTER JOURNAL 2016年第3期59卷 287-307页

作者： Exposito, Roberto R. Taboada, Guillermo L. Ramos, Sabela Tourino, Juan Doallo, Ramon Univ A Coruna Dept Elect & Syst Comp Architecture Grp Campus Elvina S-N La Coruna 15071 Spain

The advent of cloud computing technologies, which dynamically provide on-demand access to computational resources over the Internet, is offering new possibilities to many scientists and researchers. Nowadays, Infrastructure as a Service (IaaS) cloud providers can offset the increasing processing requirements of data-intensive computing applications, becoming an emerging alternative to traditional servers and clusters. In this paper, a comprehensive study of the leading public IaaS cloud platform, Amazon EC2, has been conducted in order to assess its suitability for data-intensive computing. One of the key contributions of this work is the analysis of the storage-optimized family of EC2 instances. Furthermore, this study presents a detailed analysis of both performance and cost metrics. More specifically, multiple experiments have been carried out to analyze the full I/O software stack, ranging from the low-level storage devices and cluster file systems up to real-world applications using representative data-intensive parallel codes and MapReduce-based workloads. The analysis of the experimental results has shown that data-intensive applications can benefit from tailored EC2-based virtual clusters, enabling users to obtain the highest performance and cost-effectiveness in the cloud.

关键词： data-intensive computing cloud computing Infrastructure as a Service Amazon EC2 cluster file system MapReduce

来源：评论

学校读者我要写书评

暂无评论

Multi-Tier Resource Allocation for data-intensive computing

引用

BIG data RESEARCH 2015年第3期2卷 110-116页

作者： Ryan, Thomas Lee, Young Choon Univ Sydney Sch Informat Technol Sydney NSW 2006 Australia Macquarie Univ Dept Comp N Ryde NSW 2109 Australia

As distributed computing systems are used more widely, driven by trends such as 'big data' and cloud computing, they are being used for an increasingly wide range of applications. With this massive increase in application heterogeneity, the ability to have a general purpose resource management technique that performs well in heterogeneous environments is becoming increasingly important. In this paper, we present Multi-Tier Resource Allocation (MTRA) as a novel fine-grained resource management technique for distributed systems. The core idea is based on allocating resources to individual tasks in a tiered or layered approach. To account for heterogeneity, we propose a dynamic resource allocation method that adjusts resource allocations to individual tasks on a cluster node based on resource utilisation levels. We demonstrate the efficacy of this technique in a data-intensive computing environment, MapReduce data processing framework in Hadoop YARN. Our results demonstrate that MTRA is an effective general purpose resource management technique particularly for data-intensive computing environments. On a range of MapReduce benchmarks in a Hadoop YARN environment, our MTRA technique improves performance by up to 18%. In a Facebook workload model it improves job execution times by 10% on average, and up to 56% for individual jobs. (C) 2015 Elsevier Inc. All rights reserved.

关键词： Resource allocation data-intensive computing Cloud computing Big data Application heterogeneity MapReduce

来源：评论

学校读者我要写书评

暂无评论

Challenges and Opportunities for data-intensive computing in the Cloud

引用

COMPUTER 2014年第12期47卷 82-85页

作者： Jung, Eun-Sung Kettimuthu, Rajkumar Argonne Natl Lab Math & Comp Sci Div Argonne IL 60439 USA

Now running mostly on high-performance computers, data-intensive applications pose several important challenges as they move toward cloud deployment.

关键词： cloud computing parallel processing cloud computing cloud deployment data-intensive applications data-intensive computing high-performance computers Big data Cloud computing data transfer Distributed databases Mathematical model Quality of service Argonne National Laboratory Cloud big data data-intensive computing software-defined networking

来源：评论

学校读者我要写书评

暂无评论

Alleviation of Disk I/O Contention in Virtualized Settings for data-intensive computing 2

Alleviation of Disk I/O Contention in Virtualized Settings f...

引用

International Symposium on Big data computing

作者： Malensek, Matthew Pallickara, Sangmi Lee Pallickara, Shrideep Colorado State Univ Dept Comp Sci Ft Collins CO 80523 USA

ISBN: (纸本)9780769556963

Steady growth in storage and processing capabilities has led to the accumulation of large-scale datasets that contain valuable insight into the interactions of complex systems, long- and short-term trends, and real-world phenomena. Converged infrastructure, operating on cloud deployments and private clusters, has emerged as an energy-efficient and cost-effective means of coping with these computing demands. However, increased collocation of storage and processing activities often leads to greater contention for resources in high-use situations. This issue is particularly pronounced when running distributed computations (such as MapReduce applications), because overall execution times are dependent on the completion time of the slowest task(s). In this study, we propose a framework that makes opinionated disk scheduling decisions to ensure high throughput for tasks that use I/O resources conservatively, while still maintaining the average performance of long-running batch processing operations. Our solution does not require modification of client applications or virtual machines, and we illustrate its efficacy on a cluster of 1,200 VMs with a variety of datasets that span over 1 Petabyte of information;in situations with high disk interference, our algorithm resulted in a 20% improvement in MapReduce completion times.

关键词： data-intensive computing I/O interference big data distributed I/O performance

来源：评论

学校读者我要写书评

暂无评论

Improving Load Balance for data-intensive computing on Cloud Platforms

Improving Load Balance for Data-Intensive Computing on Cloud...

引用

IEEE International Conference on Smart Cloud (IEEE SmartCloud)

作者： Dai, Wei Ibrahim, Ibrahim Bassiouni, Mostafa Univ Cent Florida Dept Elect & Comp Engn Orlando FL 32816 USA Univ Cent Florida Dept Comp Sci Orlando FL 32816 USA

ISBN: (纸本)9781509052639

Nowadays, big data problems are ubiquitous, which in turn creates huge demand for data-intensive computing. The advent of Cloud computing has made data-intensive computing much more accessible and affordable than ever before. One of the crucial issues that can significantly affect the performance of data-intensive applications is the load balance among cluster nodes. In this paper, we address the load balance problem in the context of Hadoop Distributed File System (HDFS), a widely used file system for data-intensive computing on Cloud platforms, and present an innovative replica placement policy for HDFS, which can perfectly balance the computing load among all cluster nodes in both homogeneous and heterogeneous cluster environments.

关键词： Cloud computing data-intensive computing Load Balance Replica Placement Hadoop Distributed File System Hadoop MapReduce

来源：评论

学校读者我要写书评

暂无评论

Scaling eCGA Model Building via data-intensive computing

Scaling eCGA Model Building via Data-Intensive Computing

引用

2010 IEEE World Congress on Computational Intelligence

作者： Verma, Abhishek Llora, Xavier Venkataraman, Shivaram Goldberg, David E. Campbell, Roy H. Univ Illinois Dept Comp Sci 201 N Goodwin Ave Urbana IL 61801 USA Univ Illinois Natl Ctr Supercomp Applicat Urbana IL 61801 USA Univ Illinois Dept Ind & Enterprise Syst Engn Urbana IL 61801 USA

ISBN: (纸本)9781424481262

This paper shows how the extended compact genetic algorithm can be scaled using data-intensive computing techniques such as MapReduce. Two different frameworks (Hadoop and MongoDB) are used to deploy MapReduce implementations of the compact and extended compact genetic algorithms. Results show that both are good choices to deal with large-scale problems as they can scale with the number of commodity machines, as opposed to previous efforts with other techniques that either required specialized high-performance hardware or shared memory environments.

关键词： Hadoop MapReduce MongoDB commodity machines data-intensive computing extended compact genetic algorithm genetic algorithms mathematics computing

来源：评论

学校读者我要写书评

暂无评论

Integrating data-intensive computing Systems with Biological data Analysis Frameworks 23

Integrating Data-Intensive Computing Systems with Biological...

引用

23rd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (PDP)

作者： Pedersen, Edvard Raknes, Inge Alexander Ernstsen, Martin Bongo, Lars Ailo Univ Tromso Dept Comp Sci N-9001 Tromso Norway Univ Tromso Ctr Bioinformat N-9001 Tromso Norway Univ Tromso Dept Chem Norstruct N-9001 Tromso Norway

ISBN: (纸本)9781479984909

Biological data analysis is typically implemented using a pipeline that combines many data analysis tools and meta-databases. These pipelines must scale to very large datasets, and therefore often require parallel and distributed computing. There are many infrastructure systems for data-intensive computing. However, most biological data analysis pipelines do not leverage these systems. An important challenge is therefore to integrate biological data analysis frameworks with data-intensive computing infrastructure systems. In this paper, we describe how we have extended data-intensive computing systems to support unmodified biological data analysis tools. We also describe four approaches for integrating the extended systems with biological data analysis frameworks, and discuss challenges for such integration on production platforms. Our results demonstrate how biological data analysis pipelines can benefit from infrastructure systems for data-intensive computing.

关键词： biological data processing data-intensive computing integration

来源：评论

学校读者我要写书评

暂无评论

An Improved Bayesian Inference Method for data-intensive computing

An Improved Bayesian Inference Method for Data-Intensive Com...

引用

6th International Symposium on Intelligence Computation and Applications (ISICA 2012)

作者： Ma, Feng Liu, Weiyi Yunnan Univ Sch Informat Sci & Engn Kunming Peoples R China

ISBN: (纸本)9783642342882

Recent years, data-intensive computing has become a research hotspot. It also proposed a new challenge to traditional Bayesian inference methods. It is known that, traditional Bayesian inference methods could do a good job when all data stay in a single station. However, when come to data-intensive computing, it would hard to apply to new situation directly because data often been distributed in multi stations under data intensive computing. Among different Bayesian inference methods, random algorithm often been regarded as a common and effective one. And the sampling method adopted in random algorithm would largely influence the efficiency of this random algorithm. Gibbs sampling method often been used in random algorithm for Bayesian inference. Taking all of this into consideration, an improved Bayesian inference method for data-intensive computing is developed in this paper, which first use improved Gibbs sampling method in each station to gain the suitable information, then union them together to infer the final result. The validity of this method is discussed in theory and illustrated by experiment.

关键词： data-intensive computing Bayesian Inference Gibbs sampling

来源：评论

学校读者我要写书评

暂无评论

A Survey of Semantics-Aware Performance Optimization for data-intensive computing 15

A Survey of Semantics-Aware Performance Optimization for Dat...

引用

15th Intl Conf on Dependable, Autonomic and Secure computing, 15th Intl Conf on Pervasive Intelligence and computing, 3rd Intl Conf on Big data Intelligence and computing and Cyber Science and Technology Congress(DASC/PiCom/dataCom/CyberSciTech)

作者： Rao, Bingbing Wang, Liqang Univ Cent Florida Dept Comp Sci Orlando FL 32816 USA

ISBN: (纸本)9781538619568

We are living in the era of Big data and witnessing the explosion of data. Given that the limitation of CPU and I/O in a single computer, the mainstream approach to scalability is to distribute computations among a large number of processing nodes in a cluster or cloud. This paradigm gives rise to the term of data-intensive computing, which denotes a data parallel approach to process massive volume of data. Through the efforts of different disciplines, several promising programming models and a few platforms have been proposed for data-intensive computing, such as MapReduce, Hadoop, Apache Spark and Dyrad. Even though a large body of research work has being proposed to improve overall performance of these platforms, there is still a gap between the actual performance demand and the capability of current commodity systems. This paper is aimed to provide a comprehensive understanding about current semantics-aware approaches to improve the performance of data-intensive computing. We first introduce common characteristics and paradigm shifts in the evolution of data-intensive computing, as well as contemporary programming models and technologies. We then propose four kinds of performance defects and survey the state-of-the-art semantics-aware techniques. Finally, we discuss the research challenges and opportunities in the field of semantics-aware performance optimization for data-intensive computing.

关键词： data-intensive computing Big data Performance Optimization Semantics-Aware Compiler-based MapReduce Spark Hadoop

来源：评论

学校读者我要写书评

暂无评论

Hyracks: A Flexible and Extensible Foundation for data-intensive computing

Hyracks: A Flexible and Extensible Foundation for Data-Inten...

引用

IEEE 27th International Conference on data Engineering (ICDE 2011)

作者： Borkar, Vinayak Carey, Michael Grover, Raman Onose, Nicola Vernica, Rares Univ Calif Irvine Dept Comp Sci Irvine CA 92697 USA

ISBN: (纸本)9781424489589

Hyracks is a new partitioned-parallel software platform designed to run data-intensive computations on large shared-nothing clusters of computers. Hyracks allows users to express a computation as a DAG of data operators and connectors. Operators operate on partitions of input data and produce partitions of output data, while connectors repartition operators' outputs to make the newly produced partitions available at the consuming operators. We describe the Hyracks end user model, for authors of dataflow jobs, and the extension model for users who wish to augment Hyracks' built-in library with new operator and/or connector types. We also describe our initial Hyracks implementation. Since Hyracks is in roughly the same space as the open source Hadoop platform, we compare Hyracks with Hadoop experimentally for several different kinds of use cases. The initial results demonstrate that Hyracks has significant promise as a next-generation platform for data-intensive applications.

关键词： Computational modeling Computer architecture DAG Hyracks end user model data connectors data handling data operators data-intensive computing open source Hadoop platform parallel programming partitioned-parallel software platform

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：