检索结果-内蒙古大学图书馆

approximate query processing Based on approximate Materialized View 1

23rd International Conference on Algorithms and Architectures for Parallel processing (ICA3PP)

作者： Wu, Yuhan Guo, Haifeng Yang, Donghua Li, Mengmeng Zheng, Bo Wang, Hongzhi Harbin Inst Technol Harbin Peoples R China ConDB Beijing Peoples R China

ISBN: (数字)9789819708017

ISBN: (纸本)9789819708000;9789819708017

In the context of big data, the interactive analysis database system needs to answer aggregate queries within a reasonable response time. The proposed AQP++ framework can integrate data preprocessing and AQP. It connects existing AQP engine with data preprocessing method to complete the connection between them in the process of interaction analysis. After the research on the application of materialized views in AQP++ framework, it is found that the materialized views used in the two parts of the framework both come from the accurate results of precomputation, so there's still a time bottleneck under large scale data. Based on such limitations, we proposed to use approximate materialized views for subsequent results reuse. We take the method of identifying approximate interval as an example, compared the improvement of AQP++ by using approximate materialized view, and trying different sampling methods to find better time and accurate performance results. By constructed larger samples, we compared the differences of time, space and accuracy between approximate and general materialized views in AQP++, and analyzed the reasons for the poor performance in some cases of our methods. Based on the experimental results, it proved that the use of approximate materialized view can improve the AQP++ framework, it effectively save time and storage space in the preprocessing stage, and obtain the accuracy similar to or better than the general AQP results as well.

关键词： approximate materialized view Materialized views reuse AQP plus plus optimization approximate query processing

来源：评论

学校读者我要写书评

暂无评论

Learning-Based Sample Tuning for approximate query processing in Interactive Data Exploration

引用

IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 2024年第11期36卷 6532-6546页

作者： Zhang, Hanbing Jing, Yinan He, Zhenying Zhang, Kai Wang, X. Sean Fudan Univ Sch Comp Sci Shanghai 200437 Peoples R China

For interactive data exploration, approximate query processing (AQP) is a useful approach that usually uses samples to provide a timely response for queries by trading query accuracy. Existing AQP systems often materialize samples in the memory for reuse to speed up query processing. How to tune the samples according to the workload is one of the key problems in AQP. However, since the data exploration workload is so complex that it cannot be accurately predicted, existing sample tuning approaches cannot adapt to the changing workload very well. To address this problem, this paper proposes a deep reinforcement learning-based sample tuner, RL-STuner. When tuning samples, RL-STuner considers the workload changes from a global perspective and uses a Deep Q-learning Network (DQN) model to select an optimal sample set that has the maximum utility for the current workload. In addition, this paper proposes a set of optimization mechanisms to reduce the sample tuning cost. Experimental results on both real-world and synthetic datasets show that RL-STuner outperforms the existing sample tuning approaches and achieves 1.6x-5.2x improvements on query accuracy with a low tuning cost.

关键词： Measurement Adaptation models Costs Tuners Accuracy Q-learning query processing Optimization Synthetic data approximate query processing interactive data exploration data analysis

来源：评论

学校读者我要写书评

暂无评论

When Quantum Computing Meets Database: A Hybrid Sampling Framework for approximate query processing

引用

IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 2024年第12期36卷 9532-9546页

作者： Wu, Sai Shi, Meng Zhang, Dongxiang Zhao, Junbo Yuan, Gongsheng Chen, Gang Zhejiang Univ State Key Lab Blockchain & Data Secur Hangzhou 310027 Peoples R China Zhejiang Univ Sch Software Technol Hangzhou 310027 Peoples R China

Quantum computing represents a next-generation technology in data processing, promising to transcend the limitations of traditional computation. In this paper, we undertake an early exploration of the potential integration of quantum computing with database query optimization. We introduce a pioneering hybrid classical-quantum algorithm for sampling-based approximate query processing (AQP). The core concept of the algorithm revolves around identifying rare groups, which often follow a long-tail distribution, and applying distinct sampling methodologies to normal and rare groups. By leveraging the quantum capabilities of the diffusion gate and QRAM, the algorithm defines a novel quantum sampling approach that iteratively amplifies the signals of these infrequent groups. The algorithm operates without the need for preprocessing or prior knowledge of workloads or data. It utilizes the power of quadratic acceleration to achieve well-balanced sampling across various data categories. Experimental results demonstrate that in the context of AQP, the new sampling scheme provides higher accuracy at the same sampling cost. Additionally, the benefits of quantum computing become more pronounced as query selectivity increases.

关键词： Quantum computing Quantum state Logic gates query processing Qubit Approximation algorithms Aggregates Accuracy Continents Symbols approximate query processing quantum computing sampling

来源：评论

学校读者我要写书评

暂无评论

approximate query processing with Error Guarantees 9th

Approximate Query Processing with Error Guarantees

引用

9th International Conference on Big Data Analytics (BDA)

作者： Ni, Tianjia Sugiura, Kento Ishikawa, Yoshiharu Lu, Kejing Nagoya Univ Grad Sch Informat Chikusa Ku Nagoya Aichi 4648601 Japan

ISBN: (纸本)9783030966003;9783030965990

In recent years, with the increase of data and the sophistication of analysis requirements, query processing in databases has become more important. Recently, approximate query processing (AQP) was proposed for efficiently executing database queries on big data. In this research, we focus on synopsis construction on a relational database and the query technology based on it, which is called Bounded approximate query (BAQ) proposed in 2019. BAQ is a synopsis construction method that focuses on aggregate queries using SQL, and realizes error-guaranteed query processing by grouping the dataset into the synopsis. In this paper, we point out the limitations of queries and datasets in BAQ and based on the result of experiments, we prove that the proposed method can be applied efficiently to data wider than the original BAQ with smaller synopsis within the error guarantee.

关键词： approximate query processing query processing Aggregate estimation

来源：评论

学校读者我要写书评

暂无评论

Acceleration of Synopsis Construction for Bounded approximate query processing 10th

Acceleration of Synopsis Construction for Bounded Approximat...

引用

29th International Conference on Database Systems for Advanced Applications (DASFAA)

作者： Ni, Tianjia Sugiura, Kento Ishikawa, Yoshiharu Lu, Kejing Nagoya Univ Grad Sch Informat Nagoya Aichi Japan

ISBN: (纸本)9789819609130;9789819609147

approximate query processing (AQP) has gained traction as an effective technique for executing queries on big data. Bounded approximate query processing (BAQ) is a recently proposed framework that stores a summary of an original table as a synopsis and ensures that its approximation errors remain below a user-specified threshold. Based on the BAQ framework, we proposed BAQ +/- to shrink a synopsis and improve query processing speed while guaranteeing strict error bounds. However, BAQ and BAQ +/- still have limitations in constructing synopses. They require time-consuming data sorting for each numerical attribute and cannot summarize high-cardinality categorical attributes, such as spatiotemporal data. To overcome these problems, we propose a novel framework called Hierarchical BAQ (HBAQ) and a synopsis construction method in this paper. HBAQ constructs multiple synopses based on the dimension tables of several categorical attributes and uses them to answer OLAP queries efficiently. We also introduce a new bucket definition to summarize numerical attributes effectively and support incremental updates for synopses. The experimental results show that HBAQ achieves half the construction time of BAQ with lower memory consumption. Furthermore, HBAQ can answer OLAP queries more efficiently than BAQ while ensuring strictly bounded errors.

关键词： approximate query processing query processing Aggregate estimation

来源：评论

学校读者我要写书评

暂无评论

Hippo: Accelerating Transaction processing for approximate query processing Engine with Sampling Semantics 12

Hippo: Accelerating Transaction Processing for Approximate Q...

引用

12th International Symposium on Computing and Networking Workshops

作者： Kawashima, Hideyuki Keio Univ Fujisawa Kanagawa Japan

ISBN: (纸本)9798331505356;9798331505349

approximate query processing provides an aggregated value of large-scale data or speeds up machine learning pre-processing through sampling operations. Existing work has assumed that all data is stored in storage, and it requires scanning the entire data space followed by sampling for the approximation. Our work changes this assumption. We present Hippo, a concurrency control protocol that invalidates unnecessary data items on insert operations. Our algorithm is naturally integrated with the deterministic concurrency control protocol, and it never aborts transactions, which is essentially different from optimistic or pessimistic protocols. In our evaluation, the proposed protocol demonstrated performance improvements of up to 235 times compared to the CLMD concurrency control protocol.

关键词： Transaction processing Concurrency control approximate query processing Sampling

来源：评论

学校读者我要写书评

暂无评论

Constrained approximate query processing with Error and Response Time-Bound Guarantees for Efficient Big Data Analytics 24

Constrained Approximate Query Processing with Error and Resp...

引用

33rd International Symposium on High-Performance Parallel and Distributed Computing (HPDC)

作者： Kim, Sungsoo Park, Choon Seo Lee, Taewhi Nam, Kihyuk ETRI Daejeon South Korea

ISBN: (纸本)9798400704130

approximate query processing (AQP) is a technique for obtaining approximate answers to queries over large datasets. AQP techniques trade off accuracy for speed, making them ideal for scenarios where exact answers are not required or the cost of obtaining exact answers is prohibitive. This paper proposes a novel machine learning (ML)-based AQP framework that leverages both generative and inferential ML models to improve accuracy and efficiency. The framework first constructs a generative ML model that learns the underlying data distribution and then generates synthetic data that follows the same distribution. The proposed framework also includes a mechanism for constrained approximate query processing (CAQP) with bounded errors and bounded response times. This allows users to specify the desired error bound for the results of an aggregation query. The framework then selects a subset of the synthetic data that is guaranteed to satisfy the error bound. An evaluation of the proposed framework using the Instacart benchmark dataset and queries demonstrates significant efficiency improvements in AQP compared to existing techniques.

关键词： approximate query processing

来源：评论

学校读者我要写书评

暂无评论

approximate query processing for Lambda Architecture 6

Approximate Query Processing for Lambda Architecture

引用

6th International Conference on Internet of Things, Big Data and Security (IoTBDS)

作者： Burdakov, Aleksey Grigorev, Uriy Ploutenko, Andrey Ermakov, Oleg Bauman Moscow State Tech Univ Moscow Russia Amur State Univ Blagoveshchensk Russia

ISBN: (纸本)9789897585043

The lambda architecture is widely used to implement streaming data processing systems. These systems create batch views (subsets of data) at the Serving Layer to speed up queries. This operation takes significant time. The article proposes a novel approach to lambda architecture implementation. A new method for approximate query processing in a system with Lambda Architecture (LA-AQP) significantly reduces aggregate (sum, count, avg) calculation error. This is achieved by using a new way of calculating the reading segments probabilities. The developed method is compared with the modern Sapprox method for processing large distributed data. Experiments demonstrate that LA-AQP almost equals Sapprox in terms of volume and time characteristics. The introduced accuracy measures (d-accuracy and e-accuracy) are up to two times better than Sapprox for total aggregate calculation. Aggregate values can vary greatly from segment to segment. It is shown that in this case the LA-AQP method gives a small error in the total aggregate calculation in contrast to Sapprox.

关键词： Lambda Architecture Stream processing approximate query processing Sapprox

来源：评论

学校读者我要写书评

暂无评论

Privacy-preserving approximate query processing with Differentially Private Generative Models

Privacy-preserving Approximate Query Processing with Differe...

引用

2023 IEEE International Conference on Big Data, BigData 2023

作者： Ock, Jiwon Lee, Taewhi Kim, Seongmin Smart Data Research Section Daejeon Korea Republic of Sungshin Women's University Department of Future Convergence Technology Engineering Seoul Korea Republic of

ISBN: (纸本)9798350324457

The increasing importance of data-driven decision-making across various sectors, coupled with the need for efficient large-scale data analysis while upholding privacy, prompts our exploration. Accordingly, approaches like synthetic data generation and approximate query processing have arisen. In this study, we combined differential privacy with approximate query processing for machine learning to enhance privacy. Our approach has been implemented through differentially private generative models within an approximate query processing framework, all of which safeguard data privacy. We provide assessments of synthetic data quality concerning sensitive data and the relative error in approximate query processing utilizing synthetic data. © 2023 IEEE.

关键词： approximate query processing Differential Privacy Generative Adversarial Network Synthetic Data

来源：评论

学校读者我要写书评

暂无评论

Exploiting Embedded Synopsis for Exact and approximate query processing 33rd

Exploiting Embedded Synopsis for Exact and Approximate Query...

引用

33rd International Conference on Database and Expert Systems Applications (DEXA)

作者： Yuasa, Hiroki Goda, Kazuo Kitsuregawa, Masaru Univ Tokyo Tokyo Japan

ISBN: (纸本)9783031124266;9783031124259

Organizing data in structured forms is a mainstream approach to improve the efficiency of query processing on database. For example, a B+Tree index is widely employed to limit the search space for selective queries, speeding up the query processing. In this direction, this paper proposes synopsis embedment, an idea of incorporating synopsis (i.e. precomputed summary or abstraction information) into an existing data structure. Using the embedded synopsis, the query processing may obtain an exact or approximate answer without accessing all the concerned records in the data structure. Thus, the use of the synopsis may reduce the access cost, improving the efficiency of the query processing. This paper presents a design of efficient search algorithms on the synopsis-embedded B+Tree structure and experimentally clarifies its performance superiority.

关键词： Synopsis embedment Data structure approximate query processing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：