检索结果-内蒙古大学图书馆

International Conference on Management of Data (SIGMOD)

作者： Zhang, Hao Yu, Jeffrey Xu Zhang, Yikai Zhao, Kangfei Chinese Univ Hong Kong Hong Kong Peoples R China

ISBN: (纸本)9781450392495

In this paper, we study parallel query processing with a focus on reducing the communication cost, which is the dominating factor in parallel query processing. The communication cost becomes large if the intermediate results between operators are large in intra-operator parallelism. In the existing approaches, it optimizes an SQL query by arranging relational algebra operators to reduce the total cost, where, for each operator, it involves (i) distribution of data partitioned to computing nodes by communication, and (ii) computation on computing nodes locally. The communication and computation are dealt with inside an operator and are not separable. In other words, it is difficult to avoid large intermediate results and hence reduce the communication cost. To reduce communication cost, we separate communication from computation using several new operators proposed in this paper. One is a pair operator (circle times) to pair the partitions of a relation R with the partitions of a relation S, where a partition is specified by a hash function. With the pair operator defined, we can explicitly deal with communication to deliver pairs of partitions to computing nodes. Together with circle times, we can also explicitly treat the local computation on a computing node as (op) over tilde for any RA (relational algebra) operator op. We give a merge operator ((U) over tilde), to collect all partial results from computing nodes as they are. In short, with circle times, (op) over tilde, and (U) over tilde, we are able to explicitly specify communication and computation for RA operators. Furthermore, we propose new techniques, namely, partitioning push-down and computation push-up to separate communication from computation for RA expressions. We prove that we can push-down/up for a wide range of relational expressions. We have developed a distributed system named Secco (Separate Communication from Computation) by revamping SparkSQL on Spark, and confirmed the efficiency of our ap

关键词： Database parallel query processing query Optimization OLAP

来源：评论

学校读者我要写书评

暂无评论

A parallel query processing system based on graph-based database partitioning

引用

INFORMATION SCIENCES 2019年 480卷 237-260页

作者： Nam, Yoon-Min Han, Donghyoung Kim, Min-Soo DGIST Daegu South Korea

As parallel database systems have large amounts of data to process, it is important to utilize a scalable and efficient horizontal database partitioning method. The existing partitioning methods have major drawbacks that not only cause large amounts of data redundancy but also still require expensive shuffle operations for join queries in many cases-despite their high data redundancy. We elucidate upon the drawbacks originating from the tree-based partitioning schemes and propose a novel graph-based database partitioning method called GPT that both improves the query performance and reduces data redundancy. We integrate the proposed GPT method into a parallel query processing system, Spark SQL, across all the relevant layers and modules, including the query plan generator and the scan operator. Through extensive experiments using three benchmarks, TPC-DS, IMDB and BioWarehouse, we show that GPT significantly outperforms the state-of-the-art method in terms of both storage overhead and query performance. (C) 2018 Elsevier Inc. All rights reserved.

关键词： Horizontal database partitioning Graph-based partitioning parallel query processing

来源：评论

学校读者我要写书评

暂无评论

QScheduler: A Tool for parallel query processing in Database Systems 19

QScheduler: A Tool for Parallel Query Processing in Database...

引用

19th International Conference on Engineering of Complex Computer Systems (ICECCS)

作者： Zhang, Qingfeng Li, Shanshan Xu, Jing Nankai Univ Coll Comp & Control Engn Tianjin 300071 Peoples R China

ISBN: (纸本)9781479954827

parallel query processing in database systems has been actively investigated for the past several years. When several queries execute at the same time, all parallel tasks will compete for access to the underlying I/O and computation resources and this will lead to more contention and thrashing. When a series of queries of different types run concurrently in a database system, if we change the execution sequence of these queries, the whole execution time of the query task may be different. This simple change in the execution order can directly affect the interactions among parallel queries. Hence, we argue that it is important to take the interactions among parallel queries into account. In this paper, we consider the problem of improving the performance of parallel query tasks based on a tool named QScheduler which is designed and implemented to control the query execution sequence. Our QScheduler treats the database system as a black box and decides which query to schedule preferentially according to the different algorithm. Our experimental evaluation using TPC-H benchmark shows that the QScheduler can capture the impact of complex query interactions and will improve the system performance efficiently.

关键词： parallel query processing query scheduler query interactions Performance optimization

来源：评论

学校读者我要写书评

暂无评论

GUFI: Fast, Secure File System Metadata Search for Both Privileged and Unprivileged Users

GUFI: Fast, Secure File System Metadata Search for Both Priv...

引用

International Conference for High Performance Computing, Networking, Storage and Analysis (HPC)

作者： Manno, Dominic Lee, Jason Challa, Prajwal Zheng, Qing Bonnie, David Grider, Gary Settlemyer, Bradley Los Alamos Natl Lab Los Alamos NM 87545 USA Univ Texas Arlington Arlington TX USA NVIDIA Austin TX USA

ISBN: (纸本)9781665454445

Modern High-Performance Computing (HPC) data centers routinely store massive data sets resulting in millions of directories and billions of files. To efficiently search and sift through these files and directories we present the Grand Unified File Index (GUFI), a novel file system metadata index that enables both privileged and regular users to rapidly locate and characterize data sets of interest. GUFI uses a hierarchical index that preserves file access permissions such that the index can be securely accessed by users while still enabling efficient, advanced analysis of storage system usage by cluster administrators. Compared with the current state-of-the-art indexing for file system metadata, GUFI is able to provide speedups of 1.5x to 230x for queries executed by administrators on a real production file system namespace. Queries executed by users, which typically cannot rely on cluster-wide indexing, see even greater speedups using GUFI.

关键词： parallel query processing file system metadata management file system access control index sharding

来源：评论

学校读者我要写书评

暂无评论

HYPERSONIC: A Hybrid parallelization Approach for Scalable Complex Event processing 22

HYPERSONIC: A Hybrid Parallelization Approach for Scalable C...

引用

International Conference on Management of Data (SIGMOD)

作者： Yankovitch, Maor Kolchinsky, Ilya Schuster, Assaf Technion Israel Inst Technol Haifa Israel

ISBN: (纸本)9781450392495

The ability to promptly and efficiently detect arbitrarily complex patterns in massive real-time data streams is a crucial requirement in many modern applications. The ever-growing scale of these applications and the sophistication of the patterns involved makes it imperative to employ advanced solutions that can optimize pattern detection. One of the most prominent and well-established ways to achieve the above goal is to apply complex event processing (CEP) in a parallel manner, using a multi-core machine and/or a distributed environment. However, the inherent tightly coupled nature of CEP severely limits the scalability of the parallelization methods currently available. In this paper, we introduce a novel parallelization mechanism for efficient complex event processing over data streams. This mechanism is based on a hybrid two-tier model combining multiple layers of parallelism. By employing a fine-grained load balancing model, this multi-layered approach leads to a substantial increase in event detection throughput, while at the same time reducing the latency and the memory consumption. An extensive experimental evaluation on multiple real-life datasets shows that our approach consistently outperforms state-of-the-art CEP parallelization methods by a factor of two to three orders of magnitude.

关键词： Complex Event processing parallel query processing Distributed query processing query Optimization

来源：评论

学校读者我要写书评

暂无评论

GUFI: fast, secure file system metadata search for both privileged and unprivileged users 22

GUFI: fast, secure file system metadata search for both priv...

引用

Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis

作者： Dominic Manno Jason Lee Prajwal Challa Qing Zheng David Bonnie Gary Grider Bradley Settlemyer Los Alamos National Laboratory University of Texas at Arlington NVIDIA

Modern High-Performance Computing (HPC) data centers routinely store massive data sets resulting in millions of directories and billions of files. To efficiently search and sift through these files and directories we present the Grand Unified File Index (GUFI), a novel file system metadata index that enables both privileged and regular users to rapidly locate and characterize data sets of interest. GUFI uses a hierarchical index that preserves file access permissions such that the index can be securely accessed by users while still enabling efficient, advanced analysis of storage system usage by cluster administrators. Compared with the current state-of-the-art indexing for file system metadata, GUFI is able to provide speedups of 1.5× to 230× for queries executed by administrators on a real production file system namespace. Queries executed by users, which typically cannot rely on cluster-wide indexing, see even greater speedups using GUFI.

关键词： file system metadata management index sharding parallel query processing file system access control

来源：评论

学校读者我要写书评

暂无评论

Cost Effective Load-Balancing Approach for Range-Partitioned Main-Memory Resident Data 29th

Cost Effective Load-Balancing Approach for Range-Partitioned...

引用

29th International Conference on Database and Expert Systems Applications (DEXA)

作者： Belayadi, Djahida Hidouci, Khaled-Walid Bellatreche, Ladjel Ordonez, Carlos Ecole Natl Super Informat LCSI Lab Algiers Algeria LIAS ISAE ENSMA F-86960 Futuroscope France Univ Houston Houston TX USA

ISBN: (纸本)9783319988122;9783319988115

Due to the availability of larger RAM capacity, there is a new trend bringing parallel main memory database systems with higher performance, compared to traditional DBMSs. In parallel database systems the most critical aspect is data partitioning, which significantly impacts query processing time. Specifically, unbalanced data partitioning introduces data skew, which ends up decreasing query performance if not managed. In this work, we focus on optimizing range queries, widely used in P2P, decision support systems and spatio-temporal databases. We improve the communication complexity of the state-of-the-art previous algorithm based on skip graphs, which required O(log p) messages between 2 nodes to rebalance load, resulting in a high complexity O(p log p) to rebalance load on the p nodes. With such high cost in mind, we propose to create a global view of data distribution among all processing nodes and database clients. Our main contribution is the Approximate Partitioning Vector (APV), which provides a global approximate view of data distribution to both processing nodes and database clients. A new data balancing algorithm, following a ring topology, reduces communication to 2 messages per node pair, resulting in O(1) communication complexity per node pair and O(p) globally among the p nodes. Experiments analyze the tradeoff between adjusting load balance and query performance.

关键词： Data skew Load-balancing parallel database Range-partitioning Main memory databases P2P databases parallel query processing

来源：评论

学校读者我要写书评

暂无评论

parallelizing Federated SPARQL Queries in Presence of Replicated Data 14th

Parallelizing Federated SPARQL Queries in Presence of Replic...

引用

14th International Semantic Web Conference (ESWC)

作者： Minier, Thomas Montoya, Gabriela Skaf-Molli, Hala Molli, Pascal Nantes Univ LS2N Nantes France Aalborg Univ Dept Comp Sci Aalborg Denmark

ISBN: (纸本)9783319704074;9783319704067

Federated query engines have been enhanced to exploit new data localities created by replicated data, e.g., FEDRA. However, existing replication aware federated query engines mainly focus on pruning sources during the source selection and query decomposition in order to reduce intermediate results thanks to data locality. In this paper, we implement a replication-aware parallel join operator: Pen. This operator can be used to exploit replicated data during query execution. For existing replication-aware federated query engines, this operator exploits replicated data to parallelize the execution of joins and reduce execution time. For Triple Pattern Fragment (TPF) clients, this operator exploits the availability of several TPF servers exposing the same dataset to share the load among the servers. We implemented Pen in the federated query engine FEDX with the replicated-aware source selection FEDRA and in the reference TPF client. We empirically evaluated the performance of engines extended with the Pen operator and the experimental results suggest that our extensions outperform the existing approaches in terms of execution time and balance of load among the servers, respectively.

关键词： Linked Data parallel query processing Fragment replication Federated SPARQL Queries processing Triple Pattern Fragment Load balancing

来源：评论

学校读者我要写书评

暂无评论

A Demonstration of Lusail - querying Linked Data at Scale 17

A Demonstration of Lusail - Querying Linked Data at Scale

引用

ACM International Conference on Management of Data

作者： Mansour, Essam Abdelaziz, Ibrahim Ouzzani, Mourad Aboulnaga, Ashraf Kalnis, Panos HBKU Qatar Comp Res Inst Doha Qatar King Abdullah Univ Sci & Technol Thuwal Saudi Arabia

ISBN: (纸本)9781450341974

There has been a proliferation of datasets available as interlinked RDF data accessible through SPARQL endpoints. This has led to the emergence of various applications in life science, distributed social networks, and Internet of Things that need to integrate data from multiple endpoints. We will demonstrate Lusail;a system that supports the need of emerging applications to access tens to hundreds of geo-distributed datasets. Lusail is a geo-distributed graph engine for querying linked RDF data. Lusail delivers outstanding performance using (i) a novel locality-aware query decomposition technique that minimizes the intermediate data to be accessed by the subqueries, and (ii) selectivity-awareness and parallel query execution to reduce network latency and to increase parallelism. During the demo, the audience will be able to query actually deployed RDF endpoints as well as large synthetic and real benchmarks that we have deployed in the public cloud. The demo will also show that Lusail outperforms state-of-the-art systems by orders of magnitude in terms of scalability and response time.

关键词： sparql rdf data query processing parallel query processing linked data federated sparql queries decentralized rdf graphs

来源：评论

学校读者我要写书评

暂无评论

GPU processing of theta-joins

引用

CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE 2017年第18期29卷 1-18页

作者： Bellas, Christos Gounaris, Anastasios Aristotle Univ Thessaloniki Dept Informat Thessaloniki 54124 Greece

The GPGPU paradigm has recently been employed to accelerate the processing of big amounts of data through the utilization of the massive parallelism offered by modern GPUs. To date, several techniques have been proposed for the implementation of simple select, aggregate, and equality join operations on GPUs. In this paper, we study the efficient implementation of theta-join queries between two relations using the CUDA framework. Theta-joins are notoriously slow and thus can benefit from massively parallel execution. However, their GPU-based implementation significantly differs from hash- and sort-based equality joins and needs to be carefully crafted. The implementation is driven by two main objectives. The first relates to the attainment of high efficiency in the parallelization through data reuse, which relates to the minimization of accesses to the slow global memory. The second is about the most efficient exploitation of the available memory given that, in general, it cannot hold the entire input and result. We propose a methodology for processing theta-joins on a GPU, which exploits the heterogeneous nature of GPGPU, while addressing memory limitations. Furthermore, we provide a series of implementation optimizations, which yield performance improvements of an order of magnitude.

关键词： CUDA GPGPU joins parallel query processing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：