检索结果-内蒙古大学图书馆

14th IEEE/IFIP Network Operations and Management Symposium (NOMS)

作者： Molka, Karsten Casale, Giuliano Molka, Thomas Moore, Laura Univ London Imperial Coll Sci Technol & Med Dept Comp London SW7 2AZ England SAP HANA Cloud Comp Syst Engn Belfast Antrim North Ireland

ISBN: (纸本)9781479909131

In-memory database systems are among the technological drivers of big data processing. In this paper we apply analytical modeling to enable efficient sizing of in-memory databases. We present novel response time approximations under online analytical processing workloads to model thread-level forkjoin and per-class memory occupation. We combine these approximations with a non-linear optimization program to minimize memory swapping in in-memory database clusters. We compare our approach with state-of-the-art response time approximations and trace-driven simulation using real data from an SAP HANA in-memory system and show that our optimization model is significantly more accurate than existing approaches at similar computational costs.

关键词： Optimization in-memory databases Performance Closed Queueing Networks Approximation SAP HANA

来源：评论

学校读者我要写书评

暂无评论

memory-Efficient Storing of Timestamps for Spatio-Temporal Data Management in Columnar in-memory databases 26th

Memory-Efficient Storing of Timestamps for Spatio-Temporal D...

引用

26th International Conference on Database Systems for Advanced Applications (DASFAA)

作者： Richly, Keven Univ Potsdam Hasso Plattner Inst Potsdam Germany

ISBN: (纸本)9783030731939;9783030731946

Vast amounts of spatio-temporal data are continuously accumulated through the wide distribution of location-acquisition technologies. Concerning the increased performance requirements of spatio-temporal data mining applications, in-memory database systems are used to store and process the data. As DRAM capacities are limited and expensive, the efficient utilization of the available resources is necessary. In contrast to storing the positions of moving objects, there is less focus on optimized storage concepts for the temporal component. However, it has a significant impact on the memory footprint and the overall system performance. Especially for columnar databases, the memory-efficient storing of timestamps is challenging as numerous compression approaches are optimized for contradicting data characteristics (e.g., low number of distinct values, sequences of equal values). In this paper, we present and compare different data layouts for columnar in-memory databases to store timestamps. Additionally, we propose an optimized approach for range queries with standard access ranges that uses multiple columns. We evaluate the memory consumption and performance of different compression techniques for specific access patterns. Based on the results, we introduce a workload-aware heuristic approach for the selection of performance and cost balancing data layouts. Further, we demonstrate that workload-driven optimizations for timestamps can significantly reduce the data footprint and increase the performance of spatio-temporal data management.

关键词： in-memory databases Data layout optimization Spatio-temporal data

来源：评论

学校读者我要写书评

暂无评论

A performance evaluation of in-memory databases

引用

JOURNAL OF KING SAUD UNIVERSITY-COMPUTER AND INFORMATION SCIENCES 2017年第4期29卷 520-525页

作者： Kabakus, Abdullah Talha Kara, Resul Abant Izzet Baysal Univ IT Ctr TR-14030 Bolu Turkey Duzce Univ Fac Engn Dept Comp Engn TR-81620 Duzce Turkey

The popularity of NoSQL databases has increased due to the need of (1) processing vast amount of data faster than the relational database management systems by taking the advantage of highly scalable architecture, (2) flexible (schema-free) data structure, and, (3) low latency and high performance. Despite that memory usage is not major criteria to evaluate performance of algorithms, since these databases serve the data from memory, their memory usages are also experimented alongside the time taken to complete each operation in the paper to reveal which one uses the memory most efficiently. Currently there exists over 225 NoSQL databases that provide different features and characteristics. So it is necessary to reveal which one provides better performance for different data operations. In this paper, we experiment the widely used in-memory databases to measure their performance in terms of (1) the time taken to complete operations, and (2) how efficiently they use memory during operations. As per the results reported in this paper, there is no database that provides the best performance for all data operations. It is also proved that even though a RDMS stores its data in memory, its overall performance is worse than NoSQL databases. (C) 2016 The Authors. Production and hosting by Elsevier B.V. on behalf of King Saud University.

关键词： NoSQL databases in-memory databases Database performance

来源：评论

学校读者我要写书评

暂无评论

Separated Allocator Metadata in Disaggregated in-memory databases: Friend or Foe? 36

Separated Allocator Metadata in Disaggregated In-Memory Data...

引用

36th IEEE International Parallel and Distributed Processing Symposium (IEEE IPDPS)

作者： Weisgut, Marcel Ritter, Daniel Boissier, Martin Perscheid, Michael Univ Potsdam Hasso Plattner Inst Potsdam Germany

ISBN: (纸本)9781665497473

memory allocation has a significant impact on the performance of in-memory databases. While state-of-the-art memory allocators work well in DRAM-only setups, some of their design decisions might no longer yield efficiency if data is tiered to disaggregated memory or secondary memory tiers. In this work, we study the performance impact of metadata in memory allocators and their tiering to disaggregated memory in the context of in-memory databases for the first time. We show how to separate metadata and application data by the example of jemalloc, which is widely used for data-intensive applications, and study performance effects for different workloads.

关键词： Disaggregated memory memory Allocators Metadata Separation in-memory databases

来源：评论

学校读者我要写书评

暂无评论

Scaling in-memory databases on multicores

Scaling In-Memory databases on multicores

引用

作者： Joao Paulo da Conceicao Soares NOVA University of Lisbon

学位级别：博士

Current computer systems have evolved from featuring only a single processing unit and limited RAM, in the order of kilobytes or few megabytes, to include several multicore processors, o↵ering in the order of several tens of concurrent execution contexts, and have main memory in the order of several tens to hundreds of gigabytes. This allows to keep all data of many applications in the main memory, leading to the development of in- memory databases. Compared to disk-backed databases, in-memory databases (IMDBs) are expected to provide better performance by incurring in less I/O overhead. In this dissertation, we present a scalability study of two general purpose IMDBs on multicore systems. The results show that current general purpose IMDBs do not scale on multicores, due to contention among threads running concurrent transactions. In this work, we explore di↵erent direction to overcome the scalability issues of IMDBs in multicores, while enforcing strong isolation semantics. First, we present a solution that requires no modification to either database systems or to the applications, called MacroDB. MacroDB replicates the database among several engines, using a master-slave replication scheme, where update transactions execute on the master, while read-only transactions execute on slaves. This reduces contention, al- lowing MacroDB to o↵er scalable performance under read-only workloads, while update- intensive workloads su↵er from performance loss, when compared to the standalone engine. Second, we delve into the database engine and identify the concurrency control mech- anism used by the storage sub-component as a scalability bottleneck. We then propose a new locking scheme that allows the removal of such mechanisms from the storage sub-component. This modification o↵ers performance improvement under all workloads, when compared to the standalone engine, while scalability is limited to read-only work- loads. Next we addressed the scalability limitations for update-

关键词： in-memory databases Performance Study Scalability Study Contention Se-rializability Replication Latch-Free Attribute Level Locking Read-before-Write Pattern

来源：评论

学校读者我要写书评

暂无评论

A Performance Evaluation of DRAM Access for in-memory databases

引用

IEEE ACCESS 2021年 9卷 146454-146470页

作者： Qian, Zhang Wei, Jianhao Xiang, Yiwen Xiao, Chuqiao East China Normal Univ Software Engn Inst Shanghai 200241 Peoples R China

The latest CPUs(computer cpu processors) employ multiple cores, massively superscalar pipelines, out-of-order execution of tons of instructions, and advanced SIMD capabilities, which can hide the memory access latency. And most of recent memory-oriented data structures have already benefit from these features. However, due to the complexity of data organization, these CPUs do not always work well in memory resident database systems (MMDBs), particularly regarding storing data in dynamic random-access memory (DRAM). This article studies memory-efficient data structures by analyzing the run time, access latency, cache misses, instructions per cycle (IPC), and DRAM reads (bytes). Then, we design and implement two data organization schemas in the main memory database: dispersing data block organization and clustering data block organization. Using algorithmic engineering and careful attention to internal parallelism, cache alignment can hide the memory access latency. However, we find that these data structures work well in some cases, though they have been eclipsed in the face of complex access paths. To determine the reasons, we study the impact of database techniques on memory access latency, such as data partitioning, storage models, and by processing algorithms. With the specific main memory database system, we estimate the performance of each data organization schema based on DRAM DDR4 and the latest Intel Haswell microarchitecture. In conclusion, this work will make DRAM access applicable in real-world situations by implementing the schema to systems, such as in-memory databases.

关键词： Random access memory Data structures Prefetching Organizations memory management Software Hardware DRAM access in-memory databases data structure MMDBs database techniques

来源：评论

学校读者我要写书评

暂无评论

PaMeCo join: A parallel main memory compact hash join

引用

INFORMATION SYSTEMS 2016年 58卷 105-125页

作者： Begley, Steven He, Zhen Chen, Yi-Ping Phoebe La Trobe Univ Dept Comp Sci & Informat Technol Bundoora Vic 3086 Australia

This paper presents a memory-constrained hash join algorithm (PaMeCo Join) designed to operate with main-memory column-store database systems. Whilst RAM has become more affordable and the popularity of main-memory database systems continues to grow, we recognize that RAM is a finite resource and that database systems rarely have an excess of memory available to them. Therefore, we design PaMeCo to operate within an arbitrary memory limitation by processing the input relations by parts, and by using a compact hash table that represents the contained tuples in a compact format. Coupled with a radix-clustering system that lowers memory latencies, we find that PaMeCo can offer competitive performance levels to other contemporary hash join algorithms in an unconstrained environment, while being up to three times faster than a high-performing hash join when memory constraints are applied. (C) 2015 Elsevier Ltd. All rights reserved.

关键词： in-memory databases Hash join memory constrained OLAP Column store

来源：评论

学校读者我要写书评

暂无评论

Incremental Checkpointing for Fault-Tolerant Stream Processing Systems: A Data Structure Approach

引用

IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTING 2022年第1期10卷 124-136页

作者： Lin, Chia-Yu Wang, Li-Chun Chang, Shu-Ping Natl Chiao Tung Univ Dept Elect & Comp Engn Hsinchu 300 Taiwan IBM TJ Watson Res Ctr Yorktown Hts NY 10598 USA

As the demand of high-speed stream processing grows, in-memory databases are widely used to analyze streaming data. It is challenging for in-memory systems to meet the requirements of high throughput and data persistence at the same time since data are not stored in disks. ARIES logging and command logging are two popular logging methods. In current applications, both ARIES logging and command logging are necessary. However, no checkpointing mechanism includes both the functions of ARIES logging method and command logging method. Besides, adopting ARIES logging method in an in-memory database creates high overhead. Command logging records redundant commands and has high storage cost. To address the above issues, we utilize order-irrelevant characteristics of data structure and incremental checkpointing concepts to devise a data structure based incremental checkpointing (DSIC) mechanism. DSIC mechanism is a very low overhead checkpointing approach while retaining the features of ARIES logging and command logging. DSIC mechanism reduces more than 70 percent logging time of the existing logging scheme and saves 40 percent storage costs of the existing logging scheme.

关键词： Checkpointing databases Data structures Real-time systems Servers Metadata Fault tolerance Stream processing systems incremental checkpointing in-memory databases key-value stores

来源：评论

学校读者我要写书评

暂无评论

Efficient co-processor utilization in database query processing

引用

INFORMATION SYSTEMS 2013年第8期38卷 1084-1096页

作者： Bress, Sebastian Beier, Felix Rauhe, Hannes Sattler, Kai-Uwe Schallehn, Eike Saake, Gunter Univ Magdeburg D-39016 Magdeburg Germany Ilmenau Univ Technol D-98684 Ilmenau Germany SAP AG D-69190 Walldorf Germany

Specialized processing units such as GPUs or FPGAs provide great opportunities to speed up database operations by exploiting parallelism and relieving the CPU. However, distributing a workload on suitable (co-)processors is a challenging task, because of the heterogeneous nature of a hybrid processor/co-processor system. In this paper, we present a framework that automatically learns and adapts execution models for arbitrary algorithms on any (co-)processor. Our physical optimizer uses the execution models to distribute a workload of database operators on available (co-)processing devices. We demonstrate its applicability for two common use cases in modern database systems. Additionally, we contribute an overview of GPU-co-processing approaches, an in-depth discussion of our framework's operator model, the required steps for deploying our framework in practice and the support of complex operators requiring multi-dimensional learning strategies. (C) 2013 Elsevier Ltd. All rights reserved.

关键词： Query optimization Learning-based decision model Database co-processing Modern hardware architectures in-memory databases

来源：评论

学校读者我要写书评

暂无评论

NUMA-Aware Scalable and Efficient In-memory Aggregation on Large Domains

引用

IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 2015年第4期27卷 1071-1084页

作者： Wang, Li Zhou, Minqi Zhang, Zhenjie Shan, Ming-Chien Zhou, Aoying E China Normal Univ Software Engn Inst Shanghai 200062 Peoples R China Illinois Singapore Pte Ltd Adv Digital Sci Ctr Singapore Singapore SAP Res Palo Alto CA USA

Business Intelligence (BI) is recognized as one of the most important IT applications in the coming big data era. In recent years, non-uniform memory access (NUMA) has become the de-facto architecture of multiprocessors on the new generation of enterprise servers. Such new architecture brings new challenges to optimization techniques on traditional operators in BI. Aggregation, for example, is one of the basic building blocks of BI, while its processing performance with existing hash-based algorithms scales poorly in terms of the number of cores under NUMA architecture. In this paper, we provide new solutions to tackle the problem of parallel hash-based aggregation, especially targeting at domains of extremely large cardinality. We propose a NUMA-aware radix partitioning (NaRP) method which divides the original huge relation table into subsets, without invoking expensive remote memory access between nodes of the cores. We also present a new efficient aggregation algorithm (EAA), to aggregate the partitioned data in parallel with low cache coherence miss and locking costs. Theoretical analysis as well as empirical study on an IBM X5 server prove that our proposals are at least two times faster than existing methods.

关键词： Aggregation radix-partitioning in-memory databases cache miss

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：