检索结果-内蒙古大学图书馆

On the performance limits of thread placement for array databases in non-uniform memory architectures

COMPUTING 2023年第5期105卷 1059-1075页

作者： Dominico, Simone de Almeida, Eduardo C. Zanata Alves, Marco A. Univ Fed Parana Dept Informat Curitiba Parana Brazil

array database management systems (array databases) are specialized software to streamline multi-dimensional data processing. Due to the data-hungry nature of multidimensional data applications (e.g., images and time series), array databases must ideally provide linear speedup when using a multi-processing system. However, when dealing with non-uniform memory access (NUMA) machines, array databases may require massive data movement for processing across the NUMA nodes resulting in severe performance impact. This paper investigates the performance impact of five well-known thread pinning strategies running array filtering operations in two different NUMA architectures. To identify the maximum potential performance improvement, we perform an in-width analysis evaluating all possible thread pinning combinations. Our experiments showed execution metrics of two array databases, namely SAVIME and SciDB. We observe a maximum speedup by 2.25x in SAVIME with a reduction in remote memory access by 5x . For SciDB, we observed a speedup of up to 5.83 x and a reduction on the remote memory access by 4.1x . Our main finding is that well-known static thread pinning strategies only yield 48% from the potential speedup (and 26% of the energy reduction), opening multiple opportunities for improvements.

关键词： NUMA array databases Thread placement Query Processing

来源：评论

学校读者我要写书评

暂无评论

ReSKY: Efficient Subarray Skyline Computation in array databases

引用

DISTRIBUTED AND PARALLEL databases 2022年第2-3期40卷 261-298页

作者： Choi, Dalsu Yoon, Hyunsik Chung, Yon Dohn Korea Univ Seoul South Korea

Large-scale spatial data have been generated in various fields such as scientific domains and location-based services. array databases, which model a space as an array, have become one of the means of managing such spatial data. Each cell in an array tends to interact with cells neighboring with regard to dimensions (such as latitude and longitude);therefore, instead of considering a single cell, considering a concept of subarray is required in some applications. In addition, each cell has several attribute values (such as temperature and price) to indicate its features. Based on the two observations, we propose a new type of query, subarray skyline, that provides a way to find meaningful subarrays or filter less meaningful subarrays considering attributes. We also introduce an efficient processing method, ReSKY, for subarray skyline query processing. To handle large-scale spatial data, we extend ReSKY to distributed processing. We also propose another version of ReSKY that reduces memory usage during query processing. Through extensive experiments using an array database and real datasets, we show that ReSKY has better performance than the existing techniques.

关键词： Subarray skyline array databases Distributed processing

来源：评论

学校读者我要写书评

暂无评论

NoGar: A Non-cooperative Game for Thread Pinning in array databases 34th

NoGar: A Non-cooperative Game for Thread Pinning in Array Da...

引用

34th International Conference on Database and Expert Systems Applications (DEXA)

作者： Dominico, Simone Alves, Marco A. Z. de Almeida, Eduardo C. Univ Fed Parana Curitiba Parana Brazil

ISBN: (纸本)9783031398469;9783031398476

An array database is a software that uses non-linear data structures to store and process multidimensional data, including images and time series. As multi-dimensional data applications are generally data-intensive, array databases can benefit from multi-processing systems to improve performance. However, when dealing with Non-Uniform Memory Access (NUMA) machines, the movement of massive amounts of data across NUMA nodes may result in significant performance degradation. This paper presents a mechanism for scheduling array database threads based on data movement patterns and performance monitoring information. Our scheduling mechanism uses non-cooperative game theory to determine the optimal thread placement. Threads act as decision-makers selecting the best NUMA node based on each node's remote memory access cost. We implemented and tested our mechanism on two array databases (Savime and SciDB), demonstrating improved NUMA-affinity. With Savime, we observed a maximum speedup of 1.64x and a consistent reduction of up to 2.46x in remote data access during subarray operations. With SciDB, we observed a speedup of up to 1.38x and a reduction of 1.71x in remote data access.

关键词： array databases Thread pinning Nash equilibrium Query processing

来源：评论

学校读者我要写书评

暂无评论

array databases: concepts, standards, implementations

引用

JOURNAL OF BIG DATA 2021年第1期8卷 1-61页

作者： Baumann, Peter Misev, Dimitar Merticariu, Vlad Huu, Bang Pham Jacobs Univ Large Scale Sci Informat Syst Res Grp Bremen Germany

Multi-dimensional arrays (also known as raster data or gridded data) play a key role in many, if not all science and engineering domains where they typically represent spatio-temporal sensor, image, simulation output, or statistics "datacubes". As classic database technology does not support arrays adequately, such data today are maintained mostly in silo solutions, with architectures that tend to erode and not keep up with the increasing requirements on performance and service quality. array Database systems attempt to close this gap by providing declarative query support for flexible ad-hoc analytics on large n-D arrays, similar to what SQL offers on set-oriented data, XQuery on hierarchical data, and SPARQL and CIPHER on graph data. Today, Petascale array Database installations exist, employing massive parallelism and distributed processing. Hence, questions arise about technology and standards available, usability, and overall maturity. Several papers have compared models and formalisms, and benchmarks have been undertaken as well, typically comparing two systems against each other. While each of these represent valuable research to the best of our knowledge there is no comprehensive survey combining model, query language, architecture, and practical usability, and performance aspects. The size of this comparison differentiates our study as well with 19 systems compared, four benchmarked to an extent and depth clearly exceeding previous papers in the field;for example, subsetting tests were designed in a way that systems cannot be tuned to specifically these queries. It is hoped that this gives a representative overview to all who want to immerse into the field as well as a clear guidance to those who need to choose the best suited datacube tool for their application. This article presents results of the Research Data Alliance (RDA) array Database Assessment Working Group (ADA:WG), a subgroup of the Big Data Interest Group. It has elicited the state of the art i

关键词： arrays array databases Datacubes SQL MDA OGC WCPS

来源：评论

学校读者我要写书评

暂无评论

Performance of Null Handling in array databases

Performance of Null Handling in Array Databases

引用

2023 IEEE International Conference on Big Data, BigData 2023

作者： Misev, Dimitar Rodionychev, Mikhail Baumann, Peter Constructor University School of Computer Science & Engineering Bremen Germany

ISBN: (纸本)9798350324457

array databases specialize in storage, management, and query processing on massive multidimensional array data such as satellite image time-series, weather forecast models, IoT sensor measurements, medical imaging data, etc. Some of the values in this data may be 'null' for a variety of reasons, such as unknown, known false, etc. The DBMS must handle null values correctly and efficiently. As of today, the effects of different data structures for representing null values on query processing in array databases have not been systematically studied. As a consequence, it is not clear what the optimal way for handling null values is. In this paper, we measure how four different methods for encoding null values perform across most common categories of array operations. The result is a comprehensive overview, publicly available in open source, providing relevant insights into the performance characteristics of these data structures allowing to discern which among them offers the most optimal approach for representing null values in an array processing context. © 2023 IEEE.

关键词： array databases null values performance benchmark

来源：评论

学校读者我要写书评

暂无评论

Subarray Skyline Query Processing in array databases 2021

Subarray Skyline Query Processing in Array Databases

引用

33rd International Conference on Scientific and Statistical Database Management (SSDBM)

作者： Choi, Dalsu Yoon, Hyunsik Chung, Yon Dohn Korea Univ Seoul South Korea

ISBN: (纸本)9781450384131

With the generation of large-scale spatial data in various fields, array databases that represent space as an array have become one of the means of managing spatial data. Each cell in an array tends to interact with one another;therefore, instead of considering a single cell, considering a concept of subarray is required in some applications. In addition, each cell has several attribute values to indicate its features. Based on the two observations, we propose a new type of query, subarray skyline, that provides a way to find meaningful subarrays or filter less meaningful subarrays considering attributes. We also introduce an efficient query processing method, ReSKY, in centralized and distributed settings. Through extensive experiments using an array database and real datasets, we show that ReSKY has better performance than the existing techniques.

关键词： subarray skyline array databases distributed processing

来源：评论

学校读者我要写书评

暂无评论

Supporting Data Uncertainty in array databases 15

Supporting Data Uncertainty in Array Databases

引用

ACM SIGMOD International Conference on Management of Data

作者： Peng, Liping Diao, Yanlei Univ Massachusetts Amherst MA 01003 USA

ISBN: (纸本)9781450327589

Uncertain data management has become crucial to scientific applications. Recently, array databases have gained popularity for scientific data processing due to performance benefits. In this paper, we address uncertain data management in array databases, which may involve both value uncertainty within individual tuples and position uncertainty regarding where a tuple should belong in an array given uncertain dimension attributes. Our work defines the formal semantics of array operations under both value and position uncertainty. To address the new challenge raised by position uncertainty, we propose a suite of storage and evaluation strategies for array operations, with a focus on a new scheme that bounds the overhead of querying by strategically treating tuples with large variances via replication in storage. Results from real datasets show that for common workloads, our best-performing techniques outperform alternative methods based on state-of-the-art indexes by 1.7x to 4.3x for the Subarray operation and 1 to 2 orders of magnitude for Structure-Join, at only a small storage cost.

关键词： array databases uncertainty store-multiple

来源：评论

学校读者我要写书评

暂无评论

2 mm: A new technique for sorting data

引用

THEORETICAL COMPUTER SCIENCE 2022年 910卷 68-90页

作者： Mubarak, Abbas Iqbal, Sajid Naeem, Tariq Hussain, Shafiq Inst Southern Punjab Dept Comp Sci Multan Pakistan Bahauddin Zakariya Univ Dept Comp Sci Multan Pakistan Air Univ Dept Comp Sci Islamabad Pakistan Univ Sahiwal Dept Comp Sci Sahiwal Pakistan

In computation, sorting is highly essential to access and manipulate data efficiently. Highperformance sorting algorithms have always been in demand so that computers may process data more quickly. Since the computer has become a vital tool in various domains of human life, the various researchers have investigated and presented numerous sorting algorithms to sort elements of list with minimal execution time and least space. As the volume of data increases, the urgency for efficient data processing algorithms also increases. In this novel work, we present a comparison-based sorting algorithm titled "2 mm" which is a modification of the sorting algorithm Min-Max Bidirectional Parallel Selection Sort (MMBPSS). The proposed algorithm follows divide and conquers method and uses constant space complexity i.e. O(n). Although 2 mm sort differs slightly from MMBPSS sort, its computational cost is very low and it reduces more than 50% comparison cost hence presents a more efficient solution to the sorting problems in the modern era. For performance evaluation and comparison, extensive experimentation has been performed that shows the better performance of the proposed method. (C) 2022 Elsevier B.V. All rights reserved.

关键词： Sorting Comparison based sorts Big data array databases

来源：评论

学校读者我要写书评

暂无评论

Performance Analysis of array Database Systems in Non-Uniform Memory Architecture 29

Performance Analysis of Array Database Systems in Non-Unifor...

引用

29th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP)

作者： Dominico, Simone de Almeida, Eduardo C. Alves, Marco A. Z. Meira, Jorge A. Univ Fed Parana Curitiba Parana Brazil Univ Luxembourg Luxembourg Luxembourg

ISBN: (纸本)9781665414555

array Database Management Systems (array databases) support query processing over multi-dimensional data. Data storage is implemented with non-linear structures to mitigate the shortcomings of the relational model when dealing with raw binary data, such as images, time series, and others. Due In data-hungry nature of nit:In-dimensional data applications. array databases must ideally provide a linear speedup when using a multi-processing system. When dealing with Non-Uniform Memory Access (NUMA) machines, array databases may require massive data movement across the nodes resulting in a severe performance impact. depending on the user operation. In this paper. we analyze the performance impact of the NUMA architecture in the SAVIME and SciDB array databases running five different well-known static thread pinning strategies. Our experiments showed a maximum speedup of these different strategies by 2.49x for SAVIME and up to 1.40 v. fur SciDB. We also observed that these static strategies only yield 48% from the potential speedup (and 26% of the energy reduction), opening a new research topic.

关键词： NUMA array databases Thread placement

来源：评论

学校读者我要写书评

暂无评论

MARS: A multi-level array representation for simulation data

引用

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2020年 111卷 419-434页

作者： Kim, Minsoo Suh, Ilhyun Chung, Yon Dohn Korea Univ Dept Comp Sci & Engn 145 Anam Ro Seoul South Korea Korea Univ Dept IT Convergence 145 Anam Ro Seoul South Korea

In the numerical simulation domain, owing to the huge size of data and the complexity of implementing the domain specific applications, a database-centric approach for handling multidimensional simulation data is gaining considerable attention. array databases provide an optimized set of features to support administrating multidimensional data;representing simulation data with an array can be an optimal choice. Generally, query performance on sparsely filled arrays, especially when empty cells are placed between adjacent elements, can be poor. In this context, previous studies focused on the compact representation of simulation data by reducing the number of empty cells between adjacent elements as possible. However, these methods inevitably lose the original spatial structure of elements (i.e., the relative distance and direction among elements), making it impossible to utilize the built-in multidimensional operators provided by array databases. In this paper, we propose MARS, a multi-level array representation for simulation data. MARS utilizes multiple level arrays with various resolutions to cope with the two addressed problems. In the MARS representation, elements tend to be concentrated into dense array regions, where each region is selectively stored in one of the level arrays that most reduces the empty cells between adjacent elements. Unlike existing methods, MARS retains the spatial structure of elements, and thus no additional efforts to reorganize the original spatial structure for query processing is required. We built MARS on top of SciDB and implemented a specialized command line tool for MARS. We present methods and optimized operators for query processing over MARS. We evaluate the performance of MARS using two real-world numerical simulation datasets. (C) 2019 Elsevier B.V. All rights reserved.

关键词： array databases Scientific data Query processing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：