Applications that explore, query, analyze, visualize, and, in general, process very large scale data sets are known as dataintensive Applications. Large scale dataintensive computing plays an increasingly important ...
详细信息
Applications that explore, query, analyze, visualize, and, in general, process very large scale data sets are known as dataintensive Applications. Large scale dataintensive computing plays an increasingly important role in many scientific activities and commercial applications, whether it involves data mining of commercial transactions, experimental data analysis and visualization, or intensive simulation such as climate modeling. By combining high performance computation, very large data storage, high bandwidth access, and high-speed local and wide area networking, dataintensive computing enhances the technical capabilities and usefulness of most systems. The integration of parallel and distributed computational environments will produce major improvements in performance for both computing intensive and dataintensive applications in the future. The purpose of this introductory article is to provide an overview of the main issues in parallel dataintensive computing in scientific and commercial applications and to encourage the reader to go into the more in-depth articles later in this special issue. (C) 2002 Elsevier Science B.V. All rights reserved.
Many data-intensivealgorithms-particularly in visualization, image processing, and data analysis-operate on structured data, that is, data organized in multidimensional arrays. While many of these algorithms are quit...
详细信息
ISBN:
(纸本)9781467376846
Many data-intensivealgorithms-particularly in visualization, image processing, and data analysis-operate on structured data, that is, data organized in multidimensional arrays. While many of these algorithms are quite numerically intensive, by and large, their performance is limited by the cost of memory accesses. As we move towards the exascale regime of computing, one central research challenge is finding ways to minimize data movement through the memory hierarchy, particularly within a node in a shared-memory parallel setting. We study the effects that an alternative in-memory data layout format has in terms of runtime performance gains resulting from reducing the amount of data moved through the memory hierarchy. We focus the study on shared-memory parallel implementations of two algorithms common in visualization and analysis: a stencil-based convolution kernel, which uses a structured memory access pattern, and raycasting volume rendering, which uses a semi-structured memory access pattern. The question we study is to better understand to what degree an alternative memory layout, when used by these key algorithms, will result in improved runtime performance and memory system utilization. Our approach uses a layout based on a Z-order (Morton-order) space-filling curve data organization, and we measure and report runtime and various metrics and counters associated with memory system utilization. Our results show nearly uniform improved runtime performance and improved utilization of the memory hierarchy across varying levels of concurrency the applications we tested. This approach is complementary to other memory optimization strategies like cache blocking, but may also be more general and widely applicable to a diverse set of applications.
暂无评论