检索结果-内蒙古大学图书馆

13th IEEE International Conference on Trust Security and Privacy in computing and Communications (TrustCom)

作者： Idris, Muhammad Hussain, Shujaat Lee, Sungyoung Kyung Hee Univ Ubiquitous Comp Lab Coll Elect & Informat Seoul South Korea

ISBN: (纸本)9781479965137

Hadoop based MapReduce(MR) has emerged as big data processing mechanism in terms of its data intensive applications. In data intensive systems;analysis and visualizations as a result of various algorithms can lead to differentiable and comparable results. Current implementations of MR facilitates to reuse the results of MR jobs in other MR jobs and to distribute the cloud resources among jobs. However, very little work is done in terms of using same data for multiple algorithms at the same time in a single job using either shared resources or dynamic resource allocation based on the data and scheduling of MapReduce jobs. In this paper we propose a method to execute multiple algorithms on same data in HDFS concurrently and to use the same available resources by dynamically managing the task assignment and results aggregation. Our proposed approach reduces the execution time and supports multiple algorithms execution in parallel. In-Map/In-Reduce shows 200% decrease in execution time.

关键词： Big data Hadoop MapReduce data intensive computing HDFS

来源：评论

学校读者我要写书评

暂无评论

Applying Selectively Parallel I/O Compression to Parallel Storage Systems 1

引用

20th European Conference on Parallel computing (Euro-Par)

作者： Filgueira, Rosa Atkinson, Malcolm Tanimura, Yusuke Kojima, Isao Univ Edinburgh Sch Informat Edinburgh EH8 9AB Midlothian Scotland AIST Informat Technol Res Inst Tsukuba Ibaraki Japan

ISBN: (数字)9783319098739

ISBN: (纸本)9783319098739;9783319098722

This paper presents a new I/O technique called Selectively Parallel I/O Compression (SPIOC) for providing high-speed storage and access to data in QoS enabled parallel storage systems. SPIOC reduces the time of I/O operations by applying transparent compression between the computing and the storage systems. SPIOC can predict whether to compress or not at runtime, allowing parallel or sequential compression techniques, guaranteeing QoS and allowing partial and full reading by decompressing the minimum part of the file. SPIOC maximises the measured efficiency of data movement by applying run-time customising compression before storing data in the Papio storage system.

关键词： Parallel File System data intensive computing Compression algorithms Adaptive systems

来源：评论

学校读者我要写书评

暂无评论

In-advance data Analytics for Reducing Time to Discovery 2

In-advance Data Analytics for Reducing Time to Discovery

引用

IEEE International Conference on Big data

作者： Liu, Jialin Lu, Yin Chen, Yong Texas Tech Univ Dept Comp Sci Lubbock TX 79409 USA

ISBN: (纸本)9781479956661

Scientific 2workflow involves data generation, data analysis, and knowledge discovery. As the data volume exceeds a few terabytes (TB) in a single simulation run, the data movement, which happens among data generation, data analysis, and knowledge discovery, becomes a bottleneck in most scientific big data applications. Our previous work shows that reusing the analysis results can have a significant potential in reducing the overlap between data movement among compute nodes and storage nodes. In this work, we propose a new in-advance data analytics method to augment the result reuse. The fundamental idea of this in-advance data analytics method and its prototyping system is to predict the potential useful analytics operations by studying the users' analysis pattern. The predicted analysis operation is pro-actively performed on existing data and the analysis results are stored in an in-memory database for result reuse. The evaluation shows that the in-advance data analytics method and its prototyping system gains 1.2X-6.1X speedup in I/O performance improvement with 50% data overlapping and 10%-100% operation recommendation hit rate. The proposed in-advance data analytics method brings a new promising data reduction solution for big data applications.

关键词： in-advance data analytics big data data intensive computing scientific computing

来源：评论

学校读者我要写书评

暂无评论

A general accelerated R package using GPU

A general accelerated R package using GPU

引用

IEEE International Conference on Smart City

作者： Jie Huang Bojin Zhuang Fei Su School of Information and Communication Engineering Beijing University of Posts and Telecommunications

ISBN: (纸本)9781509018949

In this paper, we present cuLib, a R package that provides an easy-to-access interface for utilizing the computing power of NVIDIA GPU. The cuLib package aims to make GPU-based parallel programming easier, flexible and high-performance. It allows the use of GPU computing in R without further knowledge because the syntax for definition and manipulation of GPU data is similar to formal R language. cuLib is compatible with device and operation system. The data interface is very flexible, enabling the users manipulate data freely. The cuLib package provides an R wrapper for libraries of NVIDIA's CUDA toolkit and numerous operations. More importantly, it is not only a mathematical tool but also practicable for the algorithm in dealing with data intensive computation.

关键词： R GPU CUDA Mathematical tool data intensive computing

来源：评论

学校读者我要写书评

暂无评论

Decreasing power consumption with energy efficient data aware strategies

引用

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2013年第5期29卷 1152-1163页

作者： Vrbsky, Susan V. Galloway, Michael Carr, Robert Nori, Rahul Grubic, David Univ Alabama Dept Comp Sci Tuscaloosa AL 35487 USA

Regardless of whether data is stored in a cluster, grid, or cloud, data management is being recognized as a significant bottleneck. computing elements can be located far away from the data storage elements. The energy efficiency of the data centers storing this data is one of the biggest issues in data intensive computing. In order to address such issues, we are designing and analyzing a series of energy efficient data aware strategies involving data replication and CPU scheduling. In this paper, we present a new strategy for data replication, called Queued Least-Frequently-Used (QLFU), and study its performance to determine if it is an energy efficient strategy. We also study the benefits of using a data aware CPU scheduling strategy, called data backfilling, which uses job preemption in order to maximize CPU usage and allows for longer periods of suspension time to save energy. We measure the performance of QLFU and existing replica strategies on a small green cluster to study the running time and power consumption of the strategies with and without data backfilling. Results from this study have demonstrated that energy efficient data management can reduce energy consumption without negatively impacting response time. (c) 2013 Elsevier B.V. All rights reserved.

关键词： data backfilling data cluster data grid data intensive computing data replication Energy Power consumption QLFU

来源：评论

学校读者我要写书评

暂无评论

Towards building a multi-datacenter infrastructure for massive remote sensing image processing

引用

CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE 2013年第12期25卷 1798-1812页

作者： Zhang, Wanfeng Wang, Lizhe Liu, Dingsheng Song, Weijing Ma, Yan Liu, Peng Chen, Dan Chinese Acad Sci Ctr Earth Observat & Digital Earth Beijing Peoples R China China Univ Geosci Sch Comp Sci Wuhan 430074 Peoples R China Chinese Acad Sci Grad Univ Beijing Peoples R China

Earth observation applications are now facing the challenges of managing and processing massive data sets from multiple sources from large-scale distributed data centers (DCs). To solve this research problem, this paper presents an infrastructure of multiple data centers (MDC) for managing and processing massive remote sensing images. The proposed system is built on both groups of distributed DCs/clusters, which are equipped with DC or cluster resource manager. Access security and information service are introduced to support this architecture of MDC. We collaboratively organized the algorithm, and data belonged to the MDC in the manner of workflow. In practice, we succeeded in working out the concrete problems regarding procedures in processing applications collaboratively and transfer the massive remote sensing dataset fast and with stable cross-MDC. On the basis of the previously mentioned research work, we will investigate the platform integration of MDC. Copyright (c) 2012 John Wiley & Sons, Ltd.

关键词： data intensive computing remote sensing image processing multi-datacenter infrastructure cloud computing

来源：评论

学校读者我要写书评

暂无评论

Big data visual analytics for exploratory earth system simulation analysis

引用

COMPUTERS & GEOSCIENCES 2013年第Dec.期61卷 71-82页

作者： Steed, Chad A. Ricciuto, Daniel M. Shipman, Galen Smith, Brian Thornton, Peter E. Wang, Dali Shi, Xiaoying Williams, Dean N. Oak Ridge Natl Lab Climate Change Sci Inst Oak Ridge TN 37831 USA Lawrence Livermore Natl Lab Livermore CA 94550 USA

Rapid increases in high performance computing are feeding the development of larger and more complex data sets in climate research, which sets the stage for so-called "big data" analysis challenges. However, conventional climate analysis techniques are inadequate in dealing with the complexities of today's data. In this paper, we describe and demonstrate a visual analytics system, called the Exploratory data analysis ENvironment (EDEN), with specific application to the analysis of complex earth system simulation data sets. EDEN represents the type of interactive visual analysis tools that are necessary to transform data into insight, thereby improving critical comprehension of earth system processes. In addition to providing an overview of EDEN, we describe real-world studies using both point ensembles and global Community Land Model Version 4 (CLM4) simulations. (C) 2013 Elsevier Ltd. All rights reserved.

关键词： Visualization Parallel coordinates Climate Sensitivity analysis data intensive computing data mining Statistical visualization Multivariate Big data

来源：评论

学校读者我要写书评

暂无评论

Decentralized Edge Clouds

引用

IEEE INTERNET computing 2013年第5期17卷 70-73页

作者： Chandra, Abhishek Weissman, Jon Heintz, Benjamin Univ Minnesota Dept Comp Sci & Engn Minneapolis MN 55455 USA Univ Minnesota Minneapolis MN 55455 USA

Nebula is a highly decentralized cloud that uses volunteer edge resources. Here, the authors provide insight into some of its key properties and design issues. They also describe a distributed MapReduce application scenario to illustrate the benefits and trade-offs of using distributed and decentralized clouds for distributed data-intensive computing applications.

关键词： Distributed databases Computational Modeling Cloud computing Computer Science Monitoring data Models Cloud Distributed Systems data intensive computing

来源：评论

学校读者我要写书评

暂无评论

Unified and Efficient HEC Storage System with a Working-Set based Reorganization Scheme

Unified and Efficient HEC Storage System with a Working-Set ...

引用

15th IEEE International Conference on Cluster computing (CLUSTER)

作者： Chen, Junjie Chen, Yong Texas Tech Univ Lubbock TX 79409 USA Texas Tech Univ Dept Comp Sci Lubbock TX 79409 USA

ISBN: (纸本)9781479908981

High-end computing (HEC) applications and simulations have become increasingly data intensive. The pressure on the storage system capability has substantially increased in recent years. Traditional hard disk drives (HDD) are dominant storage devices in HEC, but suffer seek time delays and rotational latencies. The emerged storage class memory such as Solid State Drive (SSD) provides a new promising high bandwidth and low latency storage solution, but with inherent limitations of small capacity, limited writing cycles, and high cost. SSDs and HDDs have complement characteristics in nature and there is a desire to combine and unify them to best serve HEC workloads. In this paper, we introduce our initial study of a novel Working-Set based Reorganization Scheme (WS-ROS), to manage and leverage the merits of both SSDs and HDDs and to provide a highly efficient storage system. The goal of the proposed scheme is to leverage SSDs service more read requests through keeping active data and to leverage HDDs to keep large non-active data and extend SSDs writing cycles. The preliminary evaluations have shown that the proposed scheme has significant advantages and holds a promise as a unified storage management scheme for HEC workloads.

关键词： High-end computing data intensive computing storage parallel file systems Solid State Drives

来源：评论

学校读者我要写书评

暂无评论

Hierarchical I/O Scheduling for Collective I/O 13

Hierarchical I/O Scheduling for Collective I/O

引用

13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid computing (CCGrid)

作者： Liu, Jialin Chen, Yong Zhuang, Yu Texas Tech Univ Dept Comp Sci Lubbock TX 79409 USA

ISBN: (纸本)9780769549965;9781467364652

The non-contiguous access pattern of many scientific applications results in a large number of I/O requests, which can seriously limit the data-access performance. Collective I/O has been widely used to address this issue. However, the performance of collective I/O could be dramatically degraded in today's high-performance computing system due to the increasing shuffle cost caused by highly concurrent data accesses. This situation tends to be even worse as many applications become more and more data intensive. Previous research has primarily focused on optimizing I/O access cost in collective I/O but largely ignored the shuffle cost involved. In this study, we propose a new hierarchical I/O scheduling (HIO) algorithm to address the increasing shuffle cost in collective I/O. The fundamental idea is to schedule applications' I/O requests based on a shuffle cost analysis to achieve the optimal overall performance, instead of achieving optimal I/O accesses only. The algorithm is currently evaluated with the MPICH2 and PVFS2. Both theoretical analysis and experimental tests show that the proposed hierarchical I/O scheduling has a potential in addressing the degraded performance issue of collective I/O with highly concurrent accesses.

关键词： collective I/O scheduling high-performance computing big data data intensive computing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：