检索结果-内蒙古大学图书馆

Concentric layout, a new scientific data layout for matrix data-set in Hadoop file system

INTERNATIONAL JOURNAL OF PARALLEL EMERGENT AND DISTRIBUTED SYSTEMS 2013年第5期28卷 407-433页

作者： Wang, Jun Cheng, Lu Wang, Lizhe Univ Cent Florida Dept Elect Engn & Comp Sci 4000 Cent Florida Blvd Orlando FL 32816 USA Chinese Acad Sci Ctr Earth Observat & Digital Earth Beijing 100094 Peoples R China

Due to the explosive growth in the size of scientific data-sets, data-intensive computing and analysing are an emerging trend in computational science. In these applications, data pre-processing is widely adopted because it can optimise the data layout or format beforehand to facilitate the future data access. On the other hand, current research shows an increasing popularity of MapReduce framework for large-scale data processing. However, the data access patterns which are generally applied to scientific data-set are not supported by current MapReduce framework directly. This gap motivates us to provide support for these scientific data access patterns in MapReduce framework. In our work, we study the data access patterns in matrix files and propose a new concentric data layout solution to facilitate matrix data access and analysis in MapReduce framework. Concentric data layout is a data layout which maintains the dimensional property in chunk level. Contrary to the continuous data layout adopted in the current Hadoop framework, concentric data layout stores the data from the same sub-matrix into one chunk. This layout can guarantee that the average performance of data access is optimal regardless of the various access patterns. The concentric data layout requires reorganising the data before it is being analysed or processed. Our experiments are launched on a real-world halo-finding application;the results indicate that the concentric data layout improves the overall performance by up to 38%.

关键词： data access pattern Hadoop distributed file system matrix file data layout

来源：评论

学校读者我要写书评

暂无评论

CUDA-Zero:a framework for porting shared memory GPU applications to multi-GPUs

引用

Science China(Information Sciences) 2012年第3期55卷 663-676页

作者： CHEN DeHao , CHEN WenGuang & ZHENG WeiMin Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China 1. Department of Computer Science and Technology Tsinghua University Beijing 100084 China

As the prevalence of general purpose computations on GPU, shared memory programming models were proposed to ease the pain of GPU programming. However, with the demanding needs of more intensive workloads, it’s desirable to port GPU programs to more scalable distributed memory environment, such as multi-GPUs. To achieve this, programs need to be re-written with mixed programming models (e.g. CUDA and message passing). Programmers not only need to work carefully on workload distribution, but also on scheduling mechanisms to ensure the efficiency of the execution. In this paper, we studied the possibilities of automating the process of parallelization to multi-GPUs. Starting from a GPU program written in shared memory model, our framework analyzes the access patterns of arrays in kernel functions to derive the data partition schemes. To acquire the access pattern, we proposed a 3-tiers approach: static analysis, profile based analysis and user annotation. Experiments show that most access patterns can be derived correctly by the first two tiers, which means that zero efforts are needed to port an existing application to distributed memory environment. We use our framework to parallelize several applications, and show that for certain kinds of applications, CUDA-Zero can achieve efficient parallelization in multi-GPU environment.

关键词： CUDA parallelization data access pattern multi-GPU

来源：评论

学校读者我要写书评

暂无评论

SPEEDING UP BEST NEIGHBORHOOD MATCHING ALGORITHM FOR HIGH-DEFINITION IMAGE ON GPU PLATFORM

引用

INTERNATIONAL JOURNAL OF IMAGE AND GRAPHICS 2011年第3期11卷 315-337页

作者： He, Liqiang Zhang, Guangyong Zhang, Yanyan Inner Mongolia Univ Coll Comp Sci Hohhot 010021 Inner Mongolia Peoples R China Inner Mongolia Univ Coll Comp Sci Comp architecture Hohhot 010021 Inner Mongolia Peoples R China Inner Mongolia Univ Coll Comp Sci Comp Sci & Comp architecture Hohhot 010021 Inner Mongolia Peoples R China

Error concealment restores the visual integrity of image content that has been damaged due to a bad network transmission. Best neighborhood matching (BNM) is an effective image-recovery method that exploits the information redundancy in a block-coded broken image to find similar content that it then uses to repair or conceal errors. On a high-definition image, BNM is traditionally implemented sequentially, which requires a relatively long time and thus is not suitable for real-time or high-volume use. In this paper, we analyze the data access patterns of the BNM algorithm, and exploit a graphics process unit (GPU) platform to speed up the execution through a parallel implementation. We compare and combine several different GPU optimization methods (coalesced global memory access, shared memory, register files, etc.), and propose an improvement to the parallel BNM algorithm. Experimental results show that our approach can speed up BNM 62 times over the sequential approach without any obvious loss of accuracy.

关键词： BNM CUDA data access pattern coalesced global memory access shared memory

来源：评论

学校读者我要写书评

暂无评论

Dynamic data Replication Scheme in the Cloud Computing Environment

Dynamic Data Replication Scheme in the Cloud Computing Envir...

引用

IEEE 2nd Symposium on Network Cloud Computing and Applications (NCCA)

作者： Jeon, Myunghoon Lim, Kwang-Ho Ahn, Hyun Lee, Byoung-Dai Kyonggi Univ Dept Comp Sci Suwon South Korea

ISBN: (纸本)9780769549439;9781467355810

In the cloud computing environment, data replication strategy (DRS) is used to improve data access. Related studies have proposed data replication strategies. The performances of these strategies are closely related to the users' access patterns, and work optimally for a particular data access pattern. However, as the data access patterns become more flexible and unpredictable, it is difficult to manage them with traditional replication strategies. Given this circumstance, this paper proposes an algorithm that detects changes in a user's data access pattern and dynamically applies an optimal replication strategy. The proposed algorithm has the advantage of maintaining an optimal performance by responding to various data access patterns. We tested the proposed algorithm and validated its effectiveness.

关键词： component Clouds data access pattern Dynamic Switching Replication

来源：评论

学校读者我要写书评

暂无评论

Comparative Analysis of Adaptive File Replication Algorithms for Cloud data Storage 2

Comparative Analysis of Adaptive File Replication Algorithms...

引用

2nd International Conference on Future Internet of Things and Cloud (FiCloud)

作者： Myint, Julia Hunger, Axel Univ Duisburg Essen Dept Comp Engn Duisburg Germany Univ Comp Studies Yangon Myanmar

ISBN: (纸本)9781479943579

With the popularity of cloud computing technology, storage systems becomes an important role to distribute and store massive amount of data to data centers effectively. As grow as the explosive amount of data in cloud environment day after day, data popularity in cloud computing becomes unstable and unpredictable. Moreover, maintaining the static number of replicas in the system results highly storage cost for unpopular data and inefficient for most accessed data. To address these challenges, adaptive file replication algorithm called PopStore approach is proposed by using data access history and popularity thresholds. The performance of the replication algorithms is analyzed by using Yahoo Hadoop audit log to extract data access pattern. According to the experimental results, the PopStore strategy and LALW are able to adapt the access frequency and data popularity. In addition, PopStore can save more storage cost than LALW and network usage can be saved up to 35% more than LALW strategy.

关键词： cloud storage data popularity replication data access pattern

来源：评论

学校读者我要写书评

暂无评论

Adaptive data Placement For Staging-Based Coupled Scientific Workflows 15

Adaptive Data Placement For Staging-Based Coupled Scientific...

引用

International Conference for High Performance Computing, Networking, Storage and Analysis (SC)

作者： Sun, Qian Jin, Tong Romanus, Melissa Bui, Hoang Zhang, Fan Yu, Hongfeng Kolla, Hemanth Klasky, Scott Chen, Jacqueline Parashar, Manish Rutgers State Univ Rutgers Discovery Informat Inst Piscataway NJ 08854 USA Univ Nebraska Comp Sci & Engn Lincoln NE 68588 USA Sandia Natl Labs Livermore CA 94550 USA Oak Ridge Natl Lab Oak Ridge TN 37831 USA

ISBN: (纸本)9781450337236

data staging and in-situ/in-transit data processing are emerging as attractive approaches for supporting extreme scale scientific workflows. These approaches improve end-to-end performance by enabling runtime data sharing between coupled simulations and data analytics components of the workflow. However, the complex and dynamic data exchange patterns exhibited by the workflows coupled with the varied data access behaviors make efficient data placement within the staging area challenging. In this paper, we present an adaptive data placement approach to address these challenges. Our approach adapts data placement based on application-specific dynamic data access patterns, and applies access pattern-driven and location-aware mechanisms to reduce data access costs and to support efficient data sharing between the multiple workflow components. We experimentally demonstrate the effectiveness of our approach on Titan Cray XK7 using a real combustion-analyses workflow. The evaluation results demonstrate that our approach can effectively improve data access performance and overall efficiency of coupled scientific workflows.

关键词： Adaptive data placement data access pattern in-situ/in-transit coupled scientific workflows data staging

来源：评论

学校读者我要写书评

暂无评论

pattern-Direct and Layout-Aware Replication Scheme for Parallel I/O Systems

Pattern-Direct and Layout-Aware Replication Scheme for Paral...

引用

IEEE 27th International Parallel and Distributed Processing Symposium (IPDPS)

作者： Yin, Yanlong Li, Jibing He, Jun Sun, Xian-He Thakur, Rajeev IIT Dept Comp Sci Chicago IL 60616 USA Argonne Natl Lab Math & Comp Sci Div 9700 S Cass Ave Argonne IL 60439 USA

ISBN: (纸本)9780769549712

The performance gap between computing power and the I/O system is ever increasing, and in the meantime more and more High Performance Computing (HPC) applications are becoming data intensive. This study describes an I/O data replication scheme, named pattern-Direct and Layout-Aware (PDLA) data replication scheme, to alleviate this performance gap. The basic idea of PDLA is replicating identified data access pattern, and saving these reorganized replications with optimized data layouts based on access cost analysis. A runtime system is designed and developed to integrate the PDLA replication scheme and existing parallel I/O system;a prototype of PDLA is implemented under the MPICH2 and PVFS2 environments. Experimental results show that PDLA is effective in improving data access performance of parallel I/O systems.

关键词： Parallel I/O I/O optimization data replication data reorganization data access pattern

来源：评论

学校读者我要写书评

暂无评论

An access-pattern-Aware On-Chip Vector Memory System with Automatic Loading for SIMD Architectures

An Access-Pattern-Aware On-Chip Vector Memory System with Au...

引用

IEEE High Performance Extreme Computing Conference (HPEC)

作者： Geng, Tong Diken, Erkan Wang, Tianqi Jozwiak, Lech Herbordt, Martin Boston Univ Dept Elect & Comp Engn Boston MA 02215 USA Univ Sci & Technol China Dept Phys Hefei Anhui Peoples R China Eindhoven Univ Technol Dept Elect Engn Eindhoven Netherlands

ISBN: (纸本)9781538659892

Single-Instruction-Multiple-data (SIMD) architectures are widely used to accelerate applications involving data-Level Parallelism (DLP);the on-chip memory system facilitates the communication between Processing Elements (PE) and on-chip vector memory. It is observed that inefficiency of the onchip memory system is often a computational bottleneck. In this paper, we describe the design and implementation of an efficient vector data memory system. The proposed memory system consists of two novel parts: an access-pattern-aware memory controller and an automatic loading mechanism. The memory controller reduces the data reorganization overheads. The automatic loading mechanism loads data automatically according to the access patterns without load instructions. This eliminates overhead of fetching and decoding. The proposed design is implemented and synthesized with Cadence tools. Experimental results demonstrate that our design improves the performance of 8 application kernels by 44% and reduces the energy consumption by 26%, on average.

关键词： component-SIMD Vector data Memory data access pattern Automatic Loading

来源：评论

学校读者我要写书评

暂无评论

Code restructuring for improving cache performance of MPSoCs

Code restructuring for improving cache performance of MPSoCs

引用

IEEE/ACM International Conference on Computer Aided Design

作者： Chen, G Kandemir, M Penn State Univ Dept Comp Sci & Engn University Pk PA 16802 USA

ISBN: (纸本)078039254X

One of the critical goals in code optimization for MPSoC architectures is to minimize the number of off-chip memory accesses. This is because such accesses can be extremely costly from both performance and power angles, While conventional data locality optimization techniques can be us, for improving data access pattern of each processor independently, such techniques usually do not consider locality for shared data. This paper proposes a strategy, that reduces the number of off-chip references due to shared data. It achieves this goal by restructuring a parallelized application code in such a fashion that a given data block is accessed by parallel processors within the same time frame, so that its reuse is maximized while it is in the con-chip memory space. This tends to minimize the number of off-chip references since the accesses to a given data block are clustered within a short period of time during execution. Our approach employs a polyhedral tool that helps us isolate computations that manipulate a given data block.

关键词： cache storage circuit optimisation integrated circuit design microprocessor chips parallel processing system-on-chip MPSoC architectures cache performance code restructuring data access pattern data block data locality optimization techniques off-chip me Circuit optimization system-on-chip cache storage Microprocessor chips integrated circuit design PARALLEL PROCESSING (COMPUTERS) off-chip code optimization time frame concurrent processor

来源：评论

学校读者我要写书评

暂无评论

行动无线环境下相依资料广播之排程

行动无线环境下相依资料广播之排程

引用

作者：林昆锋国立台北科技大学

学位级别：硕士

在无线行动环境下，资料广播是一个对行动用户散布资讯的一个有效的方式。在一些应用里面，所有资料被存取的关系可以被表示成一个有方向性非回圈图(Directed Acyclic Graph, DAG)。在本论文里，我们探讨在无线环境下，对于有DAG这样... 详细信息

在无线行动环境下，资料广播是一个对行动用户散布资讯的一个有效的方式。在一些应用里面，所有资料被存取的关系可以被表示成一个有方向性非回圈图(Directed Acyclic Graph, DAG)。在本论文里，我们探讨在无线环境下，对于有DAG这样的存取关系，如何有效建置一个广播排程。在单一频道里，这样一个广播排程不仅最小化存取延迟时间而且满足DAG的线性顺序。最小化存取延迟时间保证了服务的品质(QoS)。我们证明寻找最佳广播排程是一个NP-hard问题并且提出一些探索式的方法。在分析并比较这些探索式的方法之延迟时间与时间复杂度之后，我们实践并比较他们的效率。另一方面，在多广播频道里，我们也证明寻找一个有最少的延迟时间且满足祖先性质（维持DAG相依性）的最佳之广播排程是一个NP-hard问题。我们更进一步指出在特定条件下，在线性时间内对于输入的DAG可以找出一个最佳的广播排程，并以此条件提出一个演算法产生最佳广播排程。然而对于一般的DAGs而言，我们提出一些探索式的演算法来产生广播排程，并且分析比较他们的效率。

关键词： latency broadcast NP-hard data access pattern DAG

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：