检索结果-内蒙古大学图书馆

International Conference on Management of Data (SIGMOD)

作者： Zhang, Hao Yu, Jeffrey Xu Zhang, Yikai Zhao, Kangfei Chinese Univ Hong Kong Hong Kong Peoples R China

ISBN: (纸本)9781450392495

In this paper, we study parallel query processing with a focus on reducing the communication cost, which is the dominating factor in parallel query processing. The communication cost becomes large if the intermediate results between operators are large in intra-operator parallelism. In the existing approaches, it optimizes an SQL query by arranging relational algebra operators to reduce the total cost, where, for each operator, it involves (i) distribution of data partitioned to computing nodes by communication, and (ii) computation on computing nodes locally. The communication and computation are dealt with inside an operator and are not separable. In other words, it is difficult to avoid large intermediate results and hence reduce the communication cost. To reduce communication cost, we separate communication from computation using several new operators proposed in this paper. One is a pair operator (circle times) to pair the partitions of a relation R with the partitions of a relation S, where a partition is specified by a hash function. With the pair operator defined, we can explicitly deal with communication to deliver pairs of partitions to computing nodes. Together with circle times, we can also explicitly treat the local computation on a computing node as (op) over tilde for any RA (relational algebra) operator op. We give a merge operator ((U) over tilde), to collect all partial results from computing nodes as they are. In short, with circle times, (op) over tilde, and (U) over tilde, we are able to explicitly specify communication and computation for RA operators. Furthermore, we propose new techniques, namely, partitioning push-down and computation push-up to separate communication from computation for RA expressions. We prove that we can push-down/up for a wide range of relational expressions. We have developed a distributed system named Secco (Separate Communication from Computation) by revamping SparkSQL on Spark, and confirmed the efficiency of our ap

关键词： Database parallel query processing query Optimization OLAP

来源：评论

学校读者我要写书评

暂无评论

parallel query processing in databases on multicore architectures

Parallel query processing in databases on multicore architec...

引用

8th International Conference on Algorithms and Architectures for parallel processing

作者： Acker, Ralph Roth, Christian Bayer, Rudolf Transact Software Willy Brandt Allee 2 D-81829 Munich Germany Tech Univ Munich Inst Informat D-85747 Garching Germany

ISBN: (纸本)9783540695004

In this paper we present a novel and complete approach on how to encapsulate parallelism for relational database query execution that strives for maximum resource utilization for both CPU and disk activities. Its simple and robust design is capable of modeling intra- and inter-operator parallelism for one or more parallel queries in a most natural way. In addition, encapsulation guarantees that the bulk of relational operators can remain unmodified, as long as their implementation is thread-safe. We will show, that with this approach, the problem of scheduling parallel tasks is generalized, so that it can be safely entrusted to the underlying operating system (OS) without suffering any performance penalties. On the contrary, relocation of all scheduling decisions from the DBMS to the OS guarantees a centralized and therefore near-optimal resource allocation (depending on the OS's abilities) for the complete system that is hosting the database server as one of its tasks. Moreover, with this proposal, query parallelization is fully transparent on the SQL interface of the database system. Configuration of the system for effective parallel query execution can be adjusted by the DB administrator by setting two descriptive tuning parameters. A prototype implementation has been integrated into the Transbase (R) relational DBMS engine.

关键词： relational DBMS parallel query processing encapsulation intraoperator inter-operator scheduling optimization

来源：评论

学校读者我要写书评

暂无评论

A parallel query processing system based on graph-based database partitioning

引用

INFORMATION SCIENCES 2019年 480卷 237-260页

作者： Nam, Yoon-Min Han, Donghyoung Kim, Min-Soo DGIST Daegu South Korea

As parallel database systems have large amounts of data to process, it is important to utilize a scalable and efficient horizontal database partitioning method. The existing partitioning methods have major drawbacks that not only cause large amounts of data redundancy but also still require expensive shuffle operations for join queries in many cases-despite their high data redundancy. We elucidate upon the drawbacks originating from the tree-based partitioning schemes and propose a novel graph-based database partitioning method called GPT that both improves the query performance and reduces data redundancy. We integrate the proposed GPT method into a parallel query processing system, Spark SQL, across all the relevant layers and modules, including the query plan generator and the scan operator. Through extensive experiments using three benchmarks, TPC-DS, IMDB and BioWarehouse, we show that GPT significantly outperforms the state-of-the-art method in terms of both storage overhead and query performance. (C) 2018 Elsevier Inc. All rights reserved.

关键词： Horizontal database partitioning Graph-based partitioning parallel query processing

来源：评论

学校读者我要写书评

暂无评论

High performance parallel query processing on a 100 node ATM connected PC cluster

引用

IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS 1999年第1期E82D卷 54-63页

作者： Tamura, T Oguchi, M Kitsuregawa, M Mitsubishi Elect Corp Informat & Commun Syst Dev Ctr Kamakura Kanagawa 2470056 Japan Univ Tokyo Inst Ind Sci Tokyo 1068558 Japan

We developed a PC cluster system which consists of 100 PCs as a test bed for massively parallel query processing. Each PC employs the 200 MHz Pentium Pro CPU and is connected with others through an ATM switch. Because the query processing applications are insensitive to the communication latency and mainly perform integer operations, the ATM connected PC cluster approach can be considered a reasonable solution for high performance database servers with low costs. However, there has been no challenge to construct large scale PC clusters for database applications, as far as the authors know. Though we employed commodity components as much as possible. we developed the DBMS itself, because that was a key component for obtaining high performance in parallel query processing, and there seemed no system which could meet our demand. On each PC node, a server program which acts as a database kernel is running to process the queries in cooperation with other nodes. The kernel was designed to execute pipelined operators and handle voluminous data efficiently, to achieve high performance on complex decision support type queries. We used the standard benchmark, TPC-D, on a 100 GB database to verify the feasibility of our approach, through comparison of our system with commercial parallel systems. As a whole, our system exhibited sufficiently high performance which was competitive with the current TPC-D top records, in spite of not using indices. For some heavy queries in the benchmark, which have high selectivity and joinability, our system performed much better. In addition, we applied transposed file organization to the database for further performance improvement. The transposed file organization vertically partitions the tuples, enabling attribute-by-attribute access to the relations. This resulted in significant performance improvement by reducing the amount of disk I/O and shifting the bottleneck to computation.

关键词： PC cluster parallel relational database parallel query processing TPC-D benchmark

来源：评论

学校读者我要写书评

暂无评论

Extensible parallel query processing for exploratory geoscientific data mining

引用

DATA MINING AND KNOWLEDGE DISCOVERY 2001年第4期5卷 277-304页

作者： Shek, EC Muntz, RR Mesrobian, E HRL Labs LLC Informat Sci Lab Malibu CA 90265 USA Univ Calif Los Angeles Dept Comp Sci Los Angeles CA 90024 USA Disney Online Burbank CA 91521 USA

Exploratory data mining and analysis requires a computing environment which provides facilities for the user-friendly expression and rapid execution of "scientific queries." In this paper, we address research issues in the parallelization of scientific queries containing complex user-defined operations. In a parallel query execution environment, parallelizing a query execution plan involves determining how input data streams to evaluators implementing logical operations can be divided to be processed by clones of the same evaluator in parallel. We introduced the concept of "relevance window" that characterizes data lineage and data partitioning opportunities available for an user-defined evaluator. In addition, we developed a query parallelization framework by extending relational parallel query optimization algorithms to allow the parallelization characteristics of user-defined evaluators to guide the process of query parallelization in an extensible query processing environment. We demonstrated the utility of our system by performing experiments mining cyclonic activity, blocking events, and the upward wave-energy propagation features from several observational and model simulation datasets.

关键词： parallel query processing extensible user-defined operations geoscientific data mining cyclone blocking events upward wave-energy propagation

来源：评论

学校读者我要写书评

暂无评论

An adaptive parallel query processing middleware for the Grid

引用

CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE 2006年第6期18卷 621-634页

作者： Da Silva, V. F. V. Dutra, M. L. Porto, F. Schulze, B. Barbosa, A. C. de Oliveira, J. C. Natl Lab Sci Computat Dept Comp Sci BR-25651075 Petropolis RJ Brazil Pontificia Univ Catolica Rio de Janeiro Dept Informat Rio de Janeiro Brazil Mil Inst Engn Dept Syst Engn Rio De Janeiro Brazil Ecole Polytech Fed Lausanne Sch Comp & Commun Sci Database Lab CH-1015 Lausanne Switzerland Univ Fed Espirito Santo Dept Comp Sci Espirito Santo Brazil

Grid services provide an important abstract layer on top of heterogeneous components (hardware and software) that take part in a Grid environment. In this scenario, applications such as scientific visualization require access to data of non-conventional data types, such as fluid path geometry, and the evaluation of special user programs and algebraic operators, such as spatial hash-join, on these data. In order to support such applications we are developing a Configurable Data Integration Middleware System for the Grid (CoDIMS-G). CoDIMS-G provides a query execution environment adapted to the heterogeneity and variations found in a Grid environment by offering a node scheduling algorithm and an adaptive query execution strategy. The latter both adapts to performance variations in a scheduled node and deals efficiently with repetitive evaluation of a query execution plan fragment, as needed for computing a particle's, trajectory. Copyright (c) 2005 John Wiley & Sons, Ltd.

关键词： middleware Grid services database parallel query processing

来源：评论

学校读者我要写书评

暂无评论

Efficient scheduling of resources for parallel query processing on grid-based architecture

Efficient scheduling of resources for parallel query process...

引用

6th Asia-Pacific Symposium on Information and Telecommunication Technologies

作者： Soe, Khin Mar Nwe, Aye Aye Aung, Than Nwe Naing, Thinn Thu Thein, Ni Lar Univ Comp Studies Yangon Myanmar

ISBN: (纸本)4885522161

parallel query processing is an increasingly attractive option for improving the performance of database systems. It can also be important in Grid since grid technologies have enabled sophisticated interaction and data sharing between resources that may belong to different departments or organizations. The decreasing cost of computing makes parallel query processing economically viable to reduce the response time of decision support queries by using parallel execution to exploit inexpensive resources. In this paper, the architecture of resource scheduling and site selection for parallel queries which are to be executed on the Grid is proposed The main aims are to address the problems of choosing appropriate resources and matching sub plans with these resources. For this reason, decisions have to be made on allocating the available processors among a number of competing database operations running in parallel. For scheduling intra-query parallelism, the new approaches, resource-balancing based site selection algorithm which determines where to execute the upcoming operation sequence based on above architecture, and the resource assignment policy for handling the idle nodes' efficiency well, are proposed. This paper also present the architecture Of scheduling inter-query parallelism and proposed hierarchical model in order to fully exploit the available parallelism and interactions among the different queries.

关键词： grid computing parallel query processing distributed query processing

来源：评论

学校读者我要写书评

暂无评论

QScheduler: A Tool for parallel query processing in Database Systems 19

QScheduler: A Tool for Parallel Query Processing in Database...

引用

19th International Conference on Engineering of Complex Computer Systems (ICECCS)

作者： Zhang, Qingfeng Li, Shanshan Xu, Jing Nankai Univ Coll Comp & Control Engn Tianjin 300071 Peoples R China

ISBN: (纸本)9781479954827

parallel query processing in database systems has been actively investigated for the past several years. When several queries execute at the same time, all parallel tasks will compete for access to the underlying I/O and computation resources and this will lead to more contention and thrashing. When a series of queries of different types run concurrently in a database system, if we change the execution sequence of these queries, the whole execution time of the query task may be different. This simple change in the execution order can directly affect the interactions among parallel queries. Hence, we argue that it is important to take the interactions among parallel queries into account. In this paper, we consider the problem of improving the performance of parallel query tasks based on a tool named QScheduler which is designed and implemented to control the query execution sequence. Our QScheduler treats the database system as a black box and decides which query to schedule preferentially according to the different algorithm. Our experimental evaluation using TPC-H benchmark shows that the QScheduler can capture the impact of complex query interactions and will improve the system performance efficiently.

关键词： parallel query processing query scheduler query interactions Performance optimization

来源：评论

学校读者我要写书评

暂无评论

An adaptive parallel query processing middleware for the Grid

An adaptive parallel query processing middleware for the Gri...

引用

1st International Workshop on Middleware for Grid Computing held in Conjunction with the ACM/IFIP/USINIX Middleware Conference

关键词： middleware Grid services database parallel query processing

来源：评论

学校读者我要写书评

暂无评论

parallel group-by query processing in a cluster architecture

引用

COMPUTER SYSTEMS SCIENCE AND ENGINEERING 2002年第1期17卷 23-39页

作者： Taniar, D Rahayu, JW Monash Univ Sch Business Syst Clayton Vic 3800 Australia La Trobe Univ Dept Comp Sci & Comp Engn Bundoora Vic 3083 Australia

SQL queries in the real world are replete with group-by clauses. These queries are often used for strategic decision making due to the nature of group-by queries where raw information is grouped according to the designated groups and within each group aggregate functions are normally carried out. In strategic decision making, parallelization of these queries becomes necessary to speed up query response time. The trend in parallel computing is now shifted to cluster architectures. In this paper, we propose a technique for parallelization of group-by queries in cluster architectures. The need for a specialized algorithm on cluster architectures is raised due to the fact that most existing work on parallel group-by query processing is based on a shared-nothing architecture, and furthermore, they cannot be efficiently implemented in cluster platforms.

关键词： group-by queries parallel query processing parallel aggregate query processing parallel databases cluster architectures parallel database architectures

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：