检索结果-内蒙古大学图书馆

ACM SIGMOD International Conference on Management of Data

作者： Chu, Shumo Balazinska, Magdalena Suciu, Dan Univ Washington Comp Sci & Engn Seattle WA 98195 USA

ISBN: (纸本)9781450327589

Big data analytics often requires processing complex queries using massive parallelism, where the main performance metrics is the communication cost incurred during data reshuffling. In this paper, we describe a system that can compute efficiently complex join queries, including queries with cyclic joins, on a massively parallel architecture. We build on two independent lines of work for multi-join query evaluation: a communication-optimal algorithm for distributed evaluation, and a worst-case optimal algorithm for sequential evaluation. We evaluate these algorithms together, then describe novel, practical optimizations for both algorithms.

关键词： join query evaluation parallel database system

来源：评论

学校读者我要写书评

暂无评论

Accelerating Massive Astronomical Cross-Match Based on Roaring Bitmap over parallel database system 7th

Accelerating Massive Astronomical Cross-Match Based on Roari...

引用

7th Computer Science On-Line Conference (CSOC)

作者： Zhang, Jianfeng Li, Hui Chen, Mei Dai, Zhenyu Zhu, Ming Guizhou Univ Coll Comp Sci & Technol Guiyang Guizhou Peoples R China Guizhou Univ Guizhou Engn Lab ACMIS Guiyang Guizhou Peoples R China Chinese Acad Sci Natl Astron Observ Beijing Peoples R China

ISBN: (纸本)9783319911861;9783319911854

In order to reduce the large network overhead and the heavy cost of cross-match on the astronomical catalog in the database cluster, we proposed a novel method of cross-matches based on Roaring Bitmap. Firstly, we store astronomical catalog data in column-oriented storage with compression setup to reduce I/O overhead of accessing field in the parallel database system. Secondly, we create the spatial index, which maps the 2D coordinates into integer number. Then, using Roaring Bitmap convert the spatial index into a bitmap index. Finally, the received spatial range search of cross-match is translated into bitmap operations to achieve batch processing. The experiments over the real large-scale astronomical data show that the proposed method is 4 to 10 times faster than traditional method, meanwhile, only consume less than 10% of memory resource.

关键词： Cross-match Catalog Roaring Bitmap parallel database system Spatial index

来源：评论

学校读者我要写书评

暂无评论

Design and Performance Evaluation of the system Architecture in a parallel database system: SPAX 13

Design and Performance Evaluation of the System Architecture...

引用

13th International Conference on Computers and Their Applications, CATA 1998

作者： Kim, Yong-Keol Park, Yoo-Mi Jin, Seong-Il Jinwon-Park Dept. of Office Automation Taejon Medical Junior College 77-1 KayangDong DongGu Taejon Korea Republic of Real-time OS Section ETRI 161 Kajong-dong Yusong-gu Taejon Korea Republic of Dept. of Computer Science Chungnam National University 220 Kang-dong Yusong-gu Taejon Korea Republic of

ISBN: (纸本)9781618395443

The parallel database system structure is a major factor for improving the performance of a parallel database system. Shared nothing parallel database system structure has several advantages such as high scalability, high availability, and high parallelism. In this paper, we propose a system structure considering a shared nothing parallel database system for SPAX system. In order to compare between proposed architecture and predefined architecture in SPAX system, we develop the performance model and perform the simulation using DEVSIM++ simulation language. © 13th International Conference on Computers and Their Applications 1998, CATA 1998. All rights reserved.

关键词： computer architecture parallel database system performance evaluation shared nothing system

来源：评论

学校读者我要写书评

暂无评论

ParaLite: A parallel database system for Data-Intensive Workflows

引用

IEICE TRANSACTIONS ON INFORMATION AND systemS 2014年第5期E97D卷 1211-1224页

作者： Chen, Ting Taura, Kenjiro Univ Tokyo Tokyo 1130033 Japan

To better support data-intensive workflows which are typically built out of various independently developed executables, this paper proposes extensions to parallel database systems called User-Defined eXecutables (UDX) and collective queries. UDX facilitates the description of workflows by enabling seamless integrations of external executables into SQL statements without any efforts to write programs confirming to strict specifications of databases. A collective query is an SQL query whose results are distributed to multiple clients and then processed by them in parallel, using arbitrary UDX. It provides efficient parallelization of executables through the data transfer optimization algorithms that distribute query results to multiple clients, taking both communication cost and computational loads into account. We implement this concept in a system called Para Lite, a parallel database system based on a popular lightweight database SQLite. Our experiments show that Para Lite has several times higher performance over Hive for typical SQL tasks and has 10x speedup compared to a commercial DBMS for executables. In addition, this paper studies a real-world text processing workflow and builds it on top of Para Lite, Hadoop, Hive and general files. Our experiences indicate that Para Lite outperforms other systems in both productivity and performance for the workflow.

关键词： data-intensive workflow parallel database system user-defined executable collective query

来源：评论

学校读者我要写书评

暂无评论

Mining association rules from relations on a parallel NCR teradata database system

Mining association rules from relations on a parallel NCR te...

引用

International Conference on Information Technology - Coding and Computing

作者： Chung, SM Mangamuri, M Wright State Univ Dept Comp Sci & Engn Dayton OH 45435 USA

ISBN: (纸本)0769521088

Data mining from relations is becoming increasingly important with the advent of parallel database systems. In this paper, we propose a new algorithm for mining association rules from relations. The new algorithm is an enhanced version of the SETM algorithm [3], and it reduces the number of candidate itemsets considerably. We implemented and evaluated the new algorithm on a parallel NCR Teradata database system. The new algorithm is much faster than the SETM algorithm, and its performance is quite scalable.

关键词： data mining association rules parallel database system performance analysis

来源：评论

学校读者我要写书评

暂无评论

Benchmarking the DBS3 parallel query optimizer

引用

IEEE parallel & DISTRIBUTED TECHNOLOGY 1996年第2期4卷 26-40页

作者： Zait, M Florescu, D Valduriez, P IBM CORP ALMADEN RES CTRSAN JOSECA 95120 INST NATL RECH INFORMAT & AUTOMAT PROJET RODINF-78153 LE CHESNAYFRANCE

The DBS3 optimizer demonstrates solutions to cost-modeling problems posed by a rich execution model. The optimizer models various execution plan formats, including bushy and zigzag trees, and pipelining is an importan... 详细信息

关键词： AS AP Benchmarking Cost Model Optimization parallel database system Performance Measurement Pipeline And Independent parallelism Validation

来源：评论

学校读者我要写书评

暂无评论

Performance of load balancing techniques for join operations in shared-nothing database management systems

引用

JOURNAL OF parallel AND DISTRIBUTED COMPUTING 1999年第1期56卷 17-46页

作者： Hua, KA Tavanapong, W Lo, YL Univ Cent Florida Sch Comp Sci Orlando FL 32816 USA Chao Yang Univ Technol Dept Informat Management Taichung Taiwan

We investigate various load balancing approaches for hash-based join techniques popular in multicomputer-based shared-nothing database systems. When the tuples are not uniformly distributed among the hash buckets, redistribution of these buckets among the processors is necessary to maintain good system performance. Two recent load balancing techniques which rely on sampling and incremental balancing, respectively, have been shown to be more robust than conventional methods. The comparison of these two approaches, however, has not been investigated. In this study, we improve these two schemes and implement them along with a conventional method and a standard join technique which does not do load balancing on an nCUBE/2 parallel computer to compare their performance. Our experimental results indicate that the sampling technique is the better approach. To further evaluate the performance of these techniques under diverse hardware conditions, we also develop a cost model and implement a simulator to perform sensitivity analyses with respect to various hardware parameters. The simulation results show that both sampling and incremental techniques provide noticeable savings over conventional methods, with the sampling approach being more scalable in supporting very large database systems. (C) 1999 Academic Press.

关键词： parallel database system join operation performance evaluation load balancing sampling

来源：评论

学校读者我要写书评

暂无评论

GeMDA: A multidimensional data partitioning technique for multiprocessor database systems

引用

DISTRIBUTED AND parallel databaseS 2001年第3期9卷 211-236页

作者： Lo, YL Hua, KA Young, HC Chaoyang Univ Technol Dept Informat Management Wufeng 413 Taichung County Taiwan Univ Cent Florida Sch Elect Engn & Comp Sci Orlando FL 32816 USA IBM Corp Almaden Res Ctr Div Res San Jose CA 95120 USA

Several studies have repeatedly demonstrated that both the performance and scalability of a shared nothing parallel database system depend on the physical layout of data across the processing nodes of the system. Today, data is allocated in these systems using horizontal partitioning strategies. This approach has a number of drawbacks. If a query involves the partitioning attribute, then typically only a small number of the processing nodes can be used to speedup the execution of this query. On the other hand, if the predicate of a selection query includes an attribute other than the partitioning attribute, then the entire data space must be searched. Again, this results in waste of computing resources. In recent years, several multidimensional data declustering techniques have been proposed to address these problems. However, these schemes are too restrictive (e.g., FX, ECC, etc.), or optimized for a certain type of queries (e.g., DM, HCAM, etc.). In this paper, we introduce a new technique which is flexible, and performs well for general queries, We prove its optimality properties, and present experimental results showing that our scheme outperforms DM and HCAM by a significant margin.

关键词： data allocation data fragmentation parallel database system query processing system utilization

来源：评论

学校读者我要写书评

暂无评论

A FAD FOR DATA INTENSIVE APPLICATIONS

引用

IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 1992年第1期4卷 34-51页

作者： DANFORTH, S VALDURIEZ, P INST NATL RECH INFORMAT & AUTOMAT F-78153 LE CHESNAYFRANCE

FAD is a strongly-typed database programming language designed for uniformly manipulating transient and persistent data on Bubba, a parallel database system developed at MCC. This paper provides an overall description of FAD, and discusses the design rationale behind a number of its distinguishing features. Comparisons with other database programming languages are provided.

关键词： COMPLEX OBJECTS DATA MODEL database PROGRAMMING LANGUAGE parallel database system QUERY OPTIMIZATION

来源：评论

学校读者我要写书评

暂无评论

A Comparative Study of Data Processing Approaches for Text Processing Workflows

A Comparative Study of Data Processing Approaches for Text P...

引用

25th ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis (SC)

作者： Chen, Ting Taura, Kenjiro Univ Tokyo Tokyo 1138654 Japan

ISBN: (纸本)9780769549569;9781467362184

Workflows are widely used in data-intensive applications since it facilities the composition of individual executables or scripts, providing an easy-to-use parallelization to domain experts. With considerable popularity of MapReduce framework, some researchers start to develop MapReduce-enabled workflows instead of general file-based ones. Meanwhile, being commercially available for nearly two decades for large-scale data processing, parallel database systems have also gotten wide attention in the support of workflows. This paper studies three real-world text processing workflows and develops them on top of several different large data processing approaches including an open source MapReduce implementation - Hadoop, a workflow-oriented parallel database system - ParaLite, and a hybrid of MapReduce and parallel DBMS - Hive. We discuss their strength/weaknesses both in terms of programmability and performance for each workflow. Our experiences and experimental results reveal some interesting trade-offs: (1) High-level query languages (SQL of ParaLite and HiveQL of Hive) are helpful for expressing data selection, aggregation and calculation by typical executables;(2) To reuse existing NLP tools, it is often important to be able to track the association between a document and its annotation attached by the tool, for which the expressiveness of SQL is particularly useful;(3) Each system has similar performance in the execution of overall workflows because essentially performing executables takes most of the time, but some small differences could reveal some potential trade-offs that each system entails for workflows.

关键词： Data processing MapReduce parallel database system text-processing workflow

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：