检索结果-内蒙古大学图书馆

Efficient data parallel algorithms for multidimensional array operations based on the EKMR scheme for distributed memory multicomputers

引用

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 2003年第7期14卷 625-639页

作者： Lin, CY Chung, YC Liu, JS Feng Chia Univ Dept Informat Engn Taichung 407 Taiwan Natl Tsing Hua Univ Dept Comp Sci Hsinchu 300 Taiwan

array operations are useful in a large number of important scientific codes, such as molecular dynamics, finite element methods, climate modeling, atmosphere and ocean sciences, etc. In our previous work, we have proposed a scheme extended Karnaugh map representation (EKMR) for multidimensional array representation. We have shown that sequential multidimensional array operation algorithms based on the EKMR scheme have better performance than those based on the traditional matrix representation (TMR) scheme. Since parallel multidimensional array operations have been an extensively investigated problem, in this paper, we present efficient data parallel algorithms for multidimensional array operations based on the EKMR scheme for distributed memory multicomputers. In data parallel programming paradigm, in general, we distribute array elements to processors based on various distribution schemes, do local computation in each processor, and collect computation results from each processor. Based on the row, the column, and the 2D mesh distribution schemes, we design data parallel algorithms for matrix-matrix addition and matrix-matrix multiplication array operations in both TMR and EKMR schemes for multidimensional arrays. We also design data parallel algorithms for six Fortran 90 array intrinsic functions, All, Maxval, Merge, Pack, Sum, and Cshift. We compare the time of the data distribution, the local computation, and the result collection phases of these array operations based on the TMR and the EKMR schemes. The experimental results show that algorithms based on the EKMR scheme outperform those based on the TMR scheme for all test cases.

关键词： data parallel algorithm array operation multidimensional array data distribution Karnaugh map

来源：评论

学校读者我要写书评

暂无评论

Swiftarray: Accelerating Queries on multidimensional arrays

引用

Tsinghua Science and Technology 2014年第5期19卷 521-530页

作者： Yifeng Geng Xiaomeng Huang Guangwen Yang Department of Computer Science and Technology Tsinghua University Ministry of Education Key Laboratory for Earth System Modeling Tsinghua University

Scientific instruments and simulation programs are generating large amounts of multidimensional array data. Queries with value and dimension subsetting conditions are commonly used by scientists to find useful information from big array data, and data storage and indexing methods play an important role in supporting queries on multidimensional array data efficiently. In this paper, we propose Swiftarray, a new storage layout with indexing techniques to accelerate queries with value and dimension subsetting conditions. In Swiftarray, the multidimensional array is divided into blocks and each block stores sorted values. Blocks are placed in the order of a Hilbert space-filling curve to improve data locality for dimension subsetting queries. We propose a 2-D-Bin method to build an index for the blocks＇ value ranges, which is an efficient way to avoid accessing unnecessary blocks for value subsetting queries. Our evaluations show that Swiftarray surpasses the NetCDF-4 format and FastBit indexing technique for queries on multidimensional arrays.

关键词： multidimensional array indexing space-filling curve

来源：评论

学校读者我要写书评

暂无评论

Finding recursions for multidimensional arrays

引用

INFORMATION AND COMPUTATION 2002年第1期173卷 1-14页

作者： Pretzel, O Univ London Imperial Coll Sci Technol & Med London SW7 2AZ England

We produce a concise description of Sakata's algorithm for finding recursion relations that are valid for an n-dimensional array. The (new) analysis extends to limit points which are excluded by previous authors. We show that under a natural hypothesis the algorithm becomes stationary below each limit point and, in particular, constructs a full set of recursion relations for a recursive array at some finite point. The method used to justify the algorithm stresses the duality between the fundamental extension operation of Sakata and the S-polynomial operation of Buchberger. (C) 2002 Elsevier Science (USA).

关键词： multidimensional array recursive arrays recursion

来源：评论

学校读者我要写书评

暂无评论

An Efficient Encoding Scheme to Handle the Address Space Overflow for Large multidimensional arrays

引用

JOURNAL OF COMPUTERS 2013年第5期8卷 1136-1144页

作者： Ahsan, Sk. Md. Masudul Hasan, K. M. Azharul KUET Dept Comp Sci & Engn Khulna 9203 Bangladesh

We present a new implementation scheme of multidimensional array for handling large scale high dimensional datasets that grows incrementally. The scheme implements a dynamic multidimensional extendible array employing a set of two dimensional extendible arrays. The multidimensional arrays provide many advantages but it has some problems as well. The Traditional multidimensional array is not dynamic extendible. Again, if the length of dimension and number of dimension of a multidimensional array is large then the address space for the array overflows quickly. In this paper, we propose a solution against the essential problem of address space overflow for handling large scale multidimensional datasets using our implementation model. We also propose a record encoding scheme based on our model for representing relational tables using multidimensional array. We evaluate our proposed scheme by comparing with Traditional multidimensional array (TMA) for different operations and find a reasonable delay of address space overflow with no retrieval penalty. We also compare the encoded scheme with traditional scheme and find that proposed encoded scheme performs better on range retrieval for sparse array.

关键词： multidimensional array Extendible array Address space overflow Karnaugh Map MOLAP Dynamic Extension and Sparse array

来源：评论

学校读者我要写书评

暂无评论

A solution of address space overflow for large multidimensional arrays

A solution of address space overflow for large multidimensio...

引用

14th International Conference on Computer and Information Technology, ICCIT 2011

作者： Ahsan, Sk. Md. Masudul Hasan, K. M. Azharul Khulna 9203 Bangladesh

ISBN: (纸本)9781612849065

We describe a novel implementation scheme of multidimensional array for handling large scale high dimensional datasets. The scheme implements a dynamic multidimensional extendible array employing a set of two dimensional extendible arrays. The multidimensional arrays provide many advantages but it has some problems as well. The Traditional multidimensional array is not dynamic extendible. Again, if the length of dimension and number of dimension of a multidimensional array is large then the address space for the array overflows soon. In this paper, we propose a solution against the essential problem of address space overflow for handling large scale multidimensional datasets using our implementation model. We evaluate our proposed scheme by comparing with Traditional multidimensional array (TMA) for different operations and find a reasonable delay on address space overflow with no significant performance degradation. © 2011 IEEE.

关键词： Address space overflow Dynamic Extension Extendible array Karnaugh Map MOLAP multidimensional array

来源：评论

学校读者我要写书评

暂无评论

A high efficient in-place transposition scheme for multidimensional arrays

A high efficient in-place transposition scheme for multidime...

引用

3rd International Conference on Information and Computing, ICIC 2010

作者： Yuan, Jie Wu, Jian-Ping Wang, Zheng-Hua National Lab. of Parallel and Distributed Processing National University of Defense Technology Changsha China School of Computer Science National University of Defense Changsha China

ISBN: (纸本)9780769540474

This paper presents a in-place cycle-reducing method for multidimensional arrays transposition in local memory to reduce the memory R/W operations. The resulting method can significantly reduce the number of subcycles (also called following-cycles) and that of memory operations for square arrays. Taking square array 256 by 256 for example, the number of subcycles used by current available best algorithm can be reduced from 32894 to only 574, using our new method, and the memory operation number can be reduced by about 32.58%, from 98430 to 66366. Taking advantage of this new method, the elapsed time can be reduced as a result. The number of cycles and operations can also be reduced for 2D rectangle arrays with 2m rows and 2n column stored in column-major, especially n=2m, even though the improvement is not as significant as that for square arrays. The improved 2D in-place algorithm has been extended to 3D arrays, and some theoretic and numerical results are also provided. Furthermore, we develop a new in-place 3D arrays transposition algorithm. This algorithm is very important for a deep research for multidimensional arrays. Experimental results show that our algorithm can improve efficiency significantly, compared to some other 3D array transposition schemes. © 2010 IEEE.

关键词： multidimensional array Memory operation Cycle-reducing Transposition In-place transposition Transposition arrays three-dimensional array Local Memory multidimensional

来源：评论

学校读者我要写书评

暂无评论

Efficient Computing of Higher Order array Indices in Parallel using GPU

Efficient Computing of Higher Order Array Indices in Paralle...

引用

IEEE-Region-10 Symposium (TENSYMP) - Technology for Impactful Sustainable Development

作者： Hasan, K. M. Azharul Chakraborty, Sagar Khulna Univ Engn & Technol Comp Sci & Engn Dept Khulna 9203 Bangladesh

ISBN: (纸本)9781728173665

Now a days most of the scientific, medical and engineering issues require to efficiently solve problems containing huge high dimensional data. But in fact, various effective algorithms are designed considering two dimensional data. Hence, to apply those effective algorithms on higher dimensional data requires converting the higher dimensional data into 2D form and after various operations, that 2D data must be converted into its original higher dimensional structure. So, Conversion between higher dimensional data and two dimensional data has become an important research issue. Some CPU based algorithms also developed for this conversion. In fact, a CPU contains a small number of very powerful cores, whereas GPU is a huge collection of modestly powerful cores. In this paper, we have shown that the combination of CPU with GPU can very efficiently perform the conversion from higher dimensional arrays to two dimensional and vice-versa. The experimental results show that in some cases this parallel implementation gives about 100x better performance than CPU implementation only. However, we've also observed some limitations of GPU-only operations, and hence CPU-GPU combination for those conversions is highly recommended.

关键词： GPU CUDA CPU Parallel processing multidimensional array

来源：评论

学校读者我要写书评

暂无评论

Adaptive Indexing for Distributed array Processing 3

Adaptive Indexing for Distributed Array Processing

引用

3rd IEEE International Congress on Big Data

作者： Geng, Yifeng Huang, Xiaomeng Yang, Guangwen Tsinghua Univ Dept Comp Sci & Technol Beijing 100084 Peoples R China Tsinghua Univ Key Lab Earth Syst Modeling Minist Educ Beijing 100084 Peoples R China Tsinghua Univ Ctr Earth Syst Sci Beijing 100084 Peoples R China

ISBN: (纸本)9781479950577

Scientists are facing the data deluge in the scientific explorations. Big data are collected by the scientific instruments and experiments. The data are usually multidimensional arrays and stored in many files. Distributed computing techniques such as MapReduce make exploring the large datasets practical. The index is a well-known measure to shorten the query processing duration. Most of existing indexing methods need a full load of the raw data to build the index. In this paper, we proposed a distributed adaptive indexing method for the distributed array-oriented query processing. Our method does not require a full scan of the array data. For each subarray accessed by a subtask, we divide the array into multiple logical blocks with a proper block size. The normal processing routine is executed when handling a query. Meanwhile, the index for the blocks accessed by the query is built at a low cost. So the whole index grows along with processing queries. This incremental manner exploits the accessed data of historical queries and eliminates the long load procedure. The experiments show that our adaptive indexing implemented over Hadoop and Hive is effective for accelerating array-oriented query processing without introducing much overhead.

关键词： multidimensional array big data MapReduce indexing

来源：评论

学校读者我要写书评

暂无评论

SLOPE: Structural Locality-Aware Programming Model for Composing array Data Analysis 34th

SLOPE: Structural Locality-Aware Programming Model for Compo...

引用

34th International Conference on High Performance Computing (ISC High Performance)

作者： Dong, Bin Wu, Kesheng Byna, Suren Tang, Houjun Lawrence Berkeley Natl Lab 1 Cyclotron Rd Berkeley CA 94720 USA

ISBN: (纸本)9783030206567;9783030206550

MapReduce brought on the Big Data revolution. However, its impact on scientific data analyses has been limited because of fundamental limitations in its data and programming models. Scientific data is typically stored as multidimensional arrays, while MapReduce is based on key-value (KV) pairs. Applying MapReduce to analyze array-based scientific data requires a conversion of arrays to KV pairs. This conversion incurs a large storage overhead and loses structural information embedded in the array. For example, analysis operations, such as convolution, are defined on the neighbors of an array element. Accessing these neighbors is straightforward using array indexes, but requires complex and expensive operations like self-join in the KV data model. In this work, we introduce a novel `structural locality'-aware programming model (SLOPE) to compose data analysis directly on multidimensional arrays. We also develop a parallel execution engine for SLOPE to transparently partition the data, to cache intermediate results, to support in-place modification, and to recover from failures. Our evaluations with real applications show that SLOPE is over ninety thousand times faster than Apache Spark and is 38% faster than TensorFlow.

关键词： multidimensional array Programming model Structural locality Composable data analysis User-defined function arrayUDF Apache Spark TensorFlow MapReduce array cache

来源：评论

学校读者我要写书评

暂无评论

Efficient Storage Scheme for n-Dimensional Sparse array: GCRS/GCCS 13

Efficient Storage Scheme for n-Dimensional Sparse Array: GCR...

引用

International Conference on High Performance Computing and Simulation (HPCS)

作者： Shaikh, Md Abu Hanif Hasan, K. M. Azharul Khulna Univ Engn & Technol Comp Sci & Engn Dept Khulna 9203 Bangladesh

ISBN: (纸本)9781467378130

Degree of data sparsity increases with the increase of number of dimensions in high performance scientific computing. Storing and applying operations on this highly sparse multidimensional data is still a challenge for data scientists. Experts suggest special storage scheme over sparse array. In traditional sparse array storage scheme, (n+l) one dimensional arrays are necessary to store n-dimensional array. In this paper, we propose 'Generalized Row/Column Storage (GCRS/GCCS)' scheme which requires three one dimensional arrays only for storing a n-dimensional array. The superiority of the GCRS/GCCS over traditional Compressed Row/Column Storage (CRS/CCS) is shown by both theoretical analysis and experimental results. In theoretical analysis, we derive equations for space and time complexity as well as the range of usability for GCRS/GCCS. It is shown that the GCRS/GCCS scheme yields to support minimum 50% data density where as the range of usability is inversely proportional with the number of dimensions for CRS/CCS scheme. The experimental result shows that the proposed GCRS/GCCS scheme outperforms the CRS/CCS scheme with respect to space complexity, time complexity and range of usability.

关键词： Sparse array multidimensional array array Storage array Compression High Performance Computing array operations CRS CCS

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：