检索结果-内蒙古大学图书馆

array databases: concepts, standards, implementations

JOURNAL OF BIG DATA 2021年第1期8卷 1-61页

作者： Baumann, Peter Misev, Dimitar Merticariu, Vlad Huu, Bang Pham Jacobs Univ Large Scale Sci Informat Syst Res Grp Bremen Germany

Multi-dimensional arrays (also known as raster data or gridded data) play a key role in many, if not all science and engineering domains where they typically represent spatio-temporal sensor, image, simulation output, or statistics "datacubes". As classic database technology does not support arrays adequately, such data today are maintained mostly in silo solutions, with architectures that tend to erode and not keep up with the increasing requirements on performance and service quality. array Database systems attempt to close this gap by providing declarative query support for flexible ad-hoc analytics on large n-D arrays, similar to what SQL offers on set-oriented data, XQuery on hierarchical data, and SPARQL and CIPHER on graph data. Today, Petascale array Database installations exist, employing massive parallelism and distributed processing. Hence, questions arise about technology and standards available, usability, and overall maturity. Several papers have compared models and formalisms, and benchmarks have been undertaken as well, typically comparing two systems against each other. While each of these represent valuable research to the best of our knowledge there is no comprehensive survey combining model, query language, architecture, and practical usability, and performance aspects. The size of this comparison differentiates our study as well with 19 systems compared, four benchmarked to an extent and depth clearly exceeding previous papers in the field;for example, subsetting tests were designed in a way that systems cannot be tuned to specifically these queries. It is hoped that this gives a representative overview to all who want to immerse into the field as well as a clear guidance to those who need to choose the best suited datacube tool for their application. This article presents results of the Research Data Alliance (RDA) array Database Assessment Working Group (ADA:WG), a subgroup of the Big Data Interest Group. It has elicited the state of the art i

关键词： arrays array databases Datacubes SQL MDA OGC WCPS

来源：评论

学校读者我要写书评

暂无评论

On the performance limits of thread placement for array databases in non-uniform memory architectures

引用

COMPUTING 2023年第5期105卷 1059-1075页

作者： Dominico, Simone de Almeida, Eduardo C. Zanata Alves, Marco A. Univ Fed Parana Dept Informat Curitiba Parana Brazil

array database management systems (array databases) are specialized software to streamline multi-dimensional data processing. Due to the data-hungry nature of multidimensional data applications (e.g., images and time series), array databases must ideally provide linear speedup when using a multi-processing system. However, when dealing with non-uniform memory access (NUMA) machines, array databases may require massive data movement for processing across the NUMA nodes resulting in severe performance impact. This paper investigates the performance impact of five well-known thread pinning strategies running array filtering operations in two different NUMA architectures. To identify the maximum potential performance improvement, we perform an in-width analysis evaluating all possible thread pinning combinations. Our experiments showed execution metrics of two array databases, namely SAVIME and SciDB. We observe a maximum speedup by 2.25x in SAVIME with a reduction in remote memory access by 5x . For SciDB, we observed a speedup of up to 5.83 x and a reduction on the remote memory access by 4.1x . Our main finding is that well-known static thread pinning strategies only yield 48% from the potential speedup (and 26% of the energy reduction), opening multiple opportunities for improvements.

关键词： NUMA array databases Thread placement Query Processing

来源：评论

学校读者我要写书评

暂无评论

ReSKY: Efficient Subarray Skyline Computation in array databases

引用

DISTRIBUTED AND PARALLEL databases 2022年第2-3期40卷 261-298页

作者： Choi, Dalsu Yoon, Hyunsik Chung, Yon Dohn Korea Univ Seoul South Korea

Large-scale spatial data have been generated in various fields such as scientific domains and location-based services. array databases, which model a space as an array, have become one of the means of managing such spatial data. Each cell in an array tends to interact with cells neighboring with regard to dimensions (such as latitude and longitude);therefore, instead of considering a single cell, considering a concept of subarray is required in some applications. In addition, each cell has several attribute values (such as temperature and price) to indicate its features. Based on the two observations, we propose a new type of query, subarray skyline, that provides a way to find meaningful subarrays or filter less meaningful subarrays considering attributes. We also introduce an efficient processing method, ReSKY, for subarray skyline query processing. To handle large-scale spatial data, we extend ReSKY to distributed processing. We also propose another version of ReSKY that reduces memory usage during query processing. Through extensive experiments using an array database and real datasets, we show that ReSKY has better performance than the existing techniques.

关键词： Subarray skyline array databases Distributed processing

来源：评论

学校读者我要写书评

暂无评论

Flexible and efficient IR using array databases

引用

VLDB JOURNAL 2008年第1期17卷 151-168页

作者： Cornacchia, Roberto Heman, Sandor Zukowski, Marcin de Vries, Arjen P. Boncz, Peter Ctr Wiskunde & Informat NL-1098 SJ Amsterdam Netherlands

The Matrix Framework is a recent proposal by Information Retrieval (IR) researchers to flexibly represent information retrieval models and concepts in a single multidimensional array framework. We provide computational support for exactly this framework with the array database system SRAM (Sparse Relational array Mapping), that works on top of a DBMS. Information retrieval models can be specified in its comprehension-based array query language, in a way that directly corresponds to the underlying mathematical formulas. SRAM efficiently stores sparse arrays in (compressed) relational tables and translates and optimizes array queries into relational queries. In this work, we describe a number of array query optimization rules. To demonstrate their effect on text retrieval, we apply them in the TREC TeraByte track (TREC-TB) efficiency task, using the Okapi BM25 model as our example. It turns out that these optimization rules enable SRAM to automatically translate the BM25 array queries into the relational equivalent of inverted list processing including compression, score materialization and quantization, such as employed by custom-built IR systems. The use of the high-performance MonetDB/X100 relational backend, that provides transparent database compression, allows the system to achieve very fast response times with good precision and low resource usage.

关键词： information retrieval array databases query optimization database compression

来源：评论

学校读者我要写书评

暂无评论

NoGar: A Non-cooperative Game for Thread Pinning in array databases 34th

NoGar: A Non-cooperative Game for Thread Pinning in Array Da...

引用

34th International Conference on Database and Expert Systems Applications (DEXA)

作者： Dominico, Simone Alves, Marco A. Z. de Almeida, Eduardo C. Univ Fed Parana Curitiba Parana Brazil

ISBN: (纸本)9783031398469;9783031398476

An array database is a software that uses non-linear data structures to store and process multidimensional data, including images and time series. As multi-dimensional data applications are generally data-intensive, array databases can benefit from multi-processing systems to improve performance. However, when dealing with Non-Uniform Memory Access (NUMA) machines, the movement of massive amounts of data across NUMA nodes may result in significant performance degradation. This paper presents a mechanism for scheduling array database threads based on data movement patterns and performance monitoring information. Our scheduling mechanism uses non-cooperative game theory to determine the optimal thread placement. Threads act as decision-makers selecting the best NUMA node based on each node's remote memory access cost. We implemented and tested our mechanism on two array databases (Savime and SciDB), demonstrating improved NUMA-affinity. With Savime, we observed a maximum speedup of 1.64x and a consistent reduction of up to 2.46x in remote data access during subarray operations. With SciDB, we observed a speedup of up to 1.38x and a reduction of 1.71x in remote data access.

关键词： array databases Thread pinning Nash equilibrium Query processing

来源：评论

学校读者我要写书评

暂无评论

Supporting Data Uncertainty in array databases 15

Supporting Data Uncertainty in Array Databases

引用

ACM SIGMOD International Conference on Management of Data

作者： Peng, Liping Diao, Yanlei Univ Massachusetts Amherst MA 01003 USA

ISBN: (纸本)9781450327589

Uncertain data management has become crucial to scientific applications. Recently, array databases have gained popularity for scientific data processing due to performance benefits. In this paper, we address uncertain data management in array databases, which may involve both value uncertainty within individual tuples and position uncertainty regarding where a tuple should belong in an array given uncertain dimension attributes. Our work defines the formal semantics of array operations under both value and position uncertainty. To address the new challenge raised by position uncertainty, we propose a suite of storage and evaluation strategies for array operations, with a focus on a new scheme that bounds the overhead of querying by strategically treating tuples with large variances via replication in storage. Results from real datasets show that for common workloads, our best-performing techniques outperform alternative methods based on state-of-the-art indexes by 1.7x to 4.3x for the Subarray operation and 1 to 2 orders of magnitude for Structure-Join, at only a small storage cost.

关键词： array databases uncertainty store-multiple

来源：评论

学校读者我要写书评

暂无评论

Subarray Skyline Query Processing in array databases 2021

Subarray Skyline Query Processing in Array Databases

引用

33rd International Conference on Scientific and Statistical Database Management (SSDBM)

作者： Choi, Dalsu Yoon, Hyunsik Chung, Yon Dohn Korea Univ Seoul South Korea

ISBN: (纸本)9781450384131

With the generation of large-scale spatial data in various fields, array databases that represent space as an array have become one of the means of managing spatial data. Each cell in an array tends to interact with one another;therefore, instead of considering a single cell, considering a concept of subarray is required in some applications. In addition, each cell has several attribute values to indicate its features. Based on the two observations, we propose a new type of query, subarray skyline, that provides a way to find meaningful subarrays or filter less meaningful subarrays considering attributes. We also introduce an efficient query processing method, ReSKY, in centralized and distributed settings. Through extensive experiments using an array database and real datasets, we show that ReSKY has better performance than the existing techniques.

关键词： subarray skyline array databases distributed processing

来源：评论

学校读者我要写书评

暂无评论

Performance of Null Handling in array databases

Performance of Null Handling in Array Databases

引用

2023 IEEE International Conference on Big Data, BigData 2023

作者： Misev, Dimitar Rodionychev, Mikhail Baumann, Peter Constructor University School of Computer Science & Engineering Bremen Germany

ISBN: (纸本)9798350324457

array databases specialize in storage, management, and query processing on massive multidimensional array data such as satellite image time-series, weather forecast models, IoT sensor measurements, medical imaging data, etc. Some of the values in this data may be 'null' for a variety of reasons, such as unknown, known false, etc. The DBMS must handle null values correctly and efficiently. As of today, the effects of different data structures for representing null values on query processing in array databases have not been systematically studied. As a consequence, it is not clear what the optimal way for handling null values is. In this paper, we measure how four different methods for encoding null values perform across most common categories of array operations. The result is a comprehensive overview, publicly available in open source, providing relevant insights into the performance characteristics of these data structures allowing to discern which among them offers the most optimal approach for representing null values in an array processing context. © 2023 IEEE.

关键词： array databases null values performance benchmark

来源：评论

学校读者我要写书评

暂无评论

Interpolation as a Bridge Between Table and array Representations of Geofields in Spatial databases

Interpolation as a Bridge Between Table and Array Representa...

引用

11th International Conference on Beyond databases, Architectures and Structures (BDAS)

作者： Bajerski, Piotr Silesian Tech Univ Inst Informat Akad 16 PL-44101 Gliwice Poland

ISBN: (纸本)9783319184227;9783319184210

Development of database technology facilitates wider integration of diverse data types, which in turn increases opportunities to ask ad hoc queries, and gives new possibilities of declarative queries optimization. For more than a decade, work on supporting multidimensional arrays in databases has been carried out, which led to such DBMSs as rasdaman, SciDB and SciQL. However, the DBMSs lack the ability to handle queries concerning geographic phenomena varying continuously over space (called geofields) which were measured in irregularly distributed nodes (e.g. air pollution). This paper addresses this issue by presenting an extension of SQL making possible to write declarative queries referencing geofields, called geofield queries. Geofield query optimization opportunities are also shortly discussed.

关键词： Spatial databases array databases SQL GIS Geofield Coverage Interpolation

来源：评论

学校读者我要写书评

暂无评论

Big Data Analytics for Earth Sciences:the EarthServer approach

引用

International Journal of Digital Earth 2016年第1期9卷 3-29页

作者： Peter Baumann Paolo Mazzetti Joachim Ungar Roberto Barbera Damiano Barboni Alan Beccati Lorenzo Bigagli Enrico Boldrini Riccardo Bruno Antonio Calanducci Piero Campalani Oliver Clements Alex Dumitrua,Mike Grant Pasquale Herzig George Kakaletris John Laxton Panagiota Koltsida Kinga Lipskoch Alireza Rezaei Mahdiraji Simone Mantovani Vlad Merticariu Antonio Messina Dimitar Misev Stefano Natali Stefano Nativi Jelmer Oosthoek Marco Pappalardo James Passmore Angelo Pio Rossi Francesco Rundo Marcus Sen Vittorio Sorbera Don Sullivan Mario Torrisi Leonardo Trovato Maria Grazia Veratelli Sebastian Wagner Large-Scale Scientific Information Systems Jacobs UniversityBremenGermany Rasdaman GmbH BremenGermany CNR-IIA National Research Council of ItalyInstitute of Atmospheric Pollution ResearchFlorenceItaly EOX IT Services GmbH ViennaAustria Consorzio COMETA CataniaItaly Division of Catania Italian National Institute for Nuclear PhysicsCataniaItaly Department of Physics and Astronomy University of CataniaCataniaItaly MEEO S.r.l. FerraraItaly Plymouth Marine Laboratory PlymouthUK Fraunhofer IGD DarmstadtGermany Athena Research and Innovation Center in Information Communication&Knowledge Technologies AthensGreece British Geological Survey EdinburghUK Software Engineering Italia S.r.l. CataniaItaly British Geological Survey KeyworthUK NASA Ames Research Center Moffett FieldCAUSA

Big Data Analytics is an emerging field since massive storage and computing capabilities have been made available by advanced *** and Environmental sciences are likely to benefit from Big Data Analytics techniques supporting the processing of the large number of Earth Observation datasets currently acquired and generated through observations and ***,Earth Science data and applications present specificities in terms of relevance of the geospatial information,wide heterogeneity of data models and formats,and complexity of ***,Big Earth Data Analytics requires specifically tailored techniques and *** EarthServer Big Earth Data Analytics engine offers a solution for coverage-type datasets,built around a high performance array database technology,and the adoption and enhancement of standards for service interaction(OGC WCS and WCPS).The EarthServer solution,led by the collection of requirements from scientific communities and international initiatives,provides a holistic approach that ranges from query languages and scalability up to mobile access and *** result is demonstrated and validated through the development of lighthouse applications in the Marine,Geology,Atmospheric,Planetary and Cryospheric science domains.

关键词： big data Big Data Analytics array databases Earth Sciences interoperability standards

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：