检索结果-内蒙古大学图书馆

ACM SIGMOD International Conference on Management of Data

作者： Scheibli, Daniel Dinse, Christian Boehm, Alexander SAP SE Walldorf Germany

ISBN: (纸本)9781450327589

QE3D is a novel query plan visualization tool that aims at providing an intuitive and holistic view of distributed query plans executed by the SAP HANA database management system. In this demonstration, we show how its interactive, three-dimensional plan representation helps to understand and quickly identify hotspots in complex, real-world scenarios.

关键词： SQL SQL query analysis parallel database visualization

来源：评论

学校读者我要写书评

暂无评论

SQL-on-Hadoop: Full Circle Back to Shared-Nothing database Architectures

引用

PROCEEDINGS OF THE VLDB ENDOWMENT 2014年第12期7卷 1295-1306页

作者： Floratou, Avrilia Minhas, Umar Farooq Ozcan, Fatma IBM Almaden Res Ctr San Jose CA 95120 USA

SQL query processing for analytics over Hadoop data has recently gained significant traction. Among many systems providing some SQI, support over Hadoop, (live is the first native Hadoop system that uses an underlying framework such as MapReduce or Tez to process SQL-like statements. Impala, on the other hand, represents the new emerging class of SQL-on-Hadoop systems that exploit a shared-nothing parallel database architecture over Hadoop. Both systems optimize their data ingestion via columnar storage, and promote different file formats: ORC and Parquet. In this paper, we compare the performance of these two systems by conducting a set of cluster experiments using a TPC-H like benchmark and two TPC-DS inspired workloads. We also closely study the I/O efficiency of their columnar formats using a set of micro-benchmarks. Our results show that Impala is 3.3X to 4.4X faster than Hive on MapReduce and 2.1X to 2.8X than Hive on Tez for the overall TPC-H experiments. Impala is also 8.2X to 10X faster than Hive on MapReduce and about 4.3X faster than Hive on Tez for the TPC-DS inspired experiments. Through detailed analysis of experimental results, we identify the reasons for this performance gap and examine the strengths and limitations of each system.

关键词： Query processing Benchmarking Digital storage Data ingestions database architecture Micro benchmark parallel database Performance gaps Shared nothing SQL query processing Sql on hadoop

来源：评论

学校读者我要写书评

暂无评论

VQA: Vertica Query Analyzer 14

VQA: Vertica Query Analyzer

引用

ACM SIGMOD International Conference on Management of Data

作者： Simitsis, Alkis Wilkinson, Kevin Blais, Jason Walsh, Joe HP Labs Palo Alto CA 94304 USA HP Vert Campbridge MA USA

ISBN: (纸本)9781450323765

database query monitoring tools collect performance metrics, such as memory and cpu usage, while a query is executing and make them available through log files or system tables. The metrics can be used to understand and diagnose query performance issues. However, analytic queries over big data presents new challenges for query monitoring tools. A long-running query may generate tens of thousands of values so simply reporting the metrics may overwhelm the user. Second, analytic queries may be written by database novices who have trouble interpreting the metrics. Third, analytic queries may access data or processing outside the database through user-defined functions and connectors. The impact of these on query performance must be understood. Vertica Query Analyzer (VQA) is a query monitoring tool to address these challenges. VQA is both a useful tool and a research platform for query analytics. It presents query performance metrics through a variety of views and granularities. In addition, it analyzes the metrics for typical performance problems and suggests corrective actions. We demonstrate VQA using TPC-DS queries which have a wide range of query duration and complexity.

关键词： SQL SQL query monitoring SQL query analysis parallel database

来源：评论

学校读者我要写书评

暂无评论

Present Situation and Prospect of Data Warehouse Architecture under the Background of Big Data

Present Situation and Prospect of Data Warehouse Architectur...

引用

International Conference on Information Science and Cloud Computing Companion (ISCC-C)

作者： Sun, Lihua Hu, Mu Ren, Kaiyin Ren, Mingming Nanjing NARI Grp Corp Nanjing 210000 Jiangsu Peoples R China

ISBN: (纸本)9781479952458

Compared with the traditional data warehouse applications, the big data analysis is characterized by its large data size and complex query analysis. In order to design the data warehouse architecture suitable for the big data analysis, this paper analyzes and summarizes the current mainstream implementation platform-parallel database, MapReduce and the hybrid architecture based on the above-mentioned two architectures. Moreover, it presents respectively their advantages and disadvantages and describes various researches of and the author's efforts on the big data analysis to make prospects for the future study.

关键词： big data data warehouse large scale MapReduce parallel database deep analysis

来源：评论

学校读者我要写书评

暂无评论

Cluster based parallel database management system for data intensive computing

引用

中国高等学校学术文摘·计算机科学 2009年第3期3卷 302-314页

作者： Jianzhong LI Wei ZHANG School of Computer Science and Technology Harbin Institute of Technology Haribin 150001 China

This paper describes a computer-cluster based parallel database management system (DBMS), InfiniteDB, developed by the authors. InfiniteDB aims at efficiently sup-port data intensive computing in response to the rapid grow-ing in database size and the need of high performance ana-lyzing of massive databases. It can be efficiently executed in the computing system composed by thousands of computers such as cloud computing system. It supports the parallelisms of intra-query, inter-query, intra-operation, inter-operation and pipelining. It provides effective strategies for managing massive databases including the multiple data declustering methods, the declustering-aware algorithms for relational operations and other database operations, and the adaptive query optimization method. It also provides the functions of parallel data warehousing and data mining, the coordinator-wrapper mechanism to support the integration of heteroge-neous information resources on the Internet, and the fault tol-erant and resilient infrastructures. It has been used in many applications and has proved quite effective for data intensive computing.

关键词： parallel database cloud computing data inten-sive super computing

来源：评论

学校读者我要写书评

暂无评论

Efficient SPARQL Query Evaluation In a database Cluster

Efficient SPARQL Query Evaluation In a Database Cluster

引用

IEEE International Congress on Big Data

作者： Du, Fang Bian, Haoqiong Chen, Yueguo Du, Xiaoyong Renmin Univ China Sch Informat Beijing Peoples R China

ISBN: (纸本)9780769550060

Efficient SPARQL query evaluation is a significant challenge when the database contains billions of RDF triples, which is very common for many existing Web-scale RDF data sources. We address this challenge by 1) effectively partitioning the whole RDF dataset into small partitions according to the schemas of the RDF subjects, and 2) elaborately placing the partitions within clusters so that, on each local partition, we can make the most advantage of the state-of-the-art SPARQL query processing engine, and across the partitions, we can exploit the power of parallel databases for achieving scalable query evaluation of massive RDF data. This paper introduces the data partitioning and placement strategies, as well as the SPARQL query evaluation and optimization techniques in a cluster environment. Experiments are conducted over a synthesized dataset and a real dataset containing billions of triples. The results demonstrate that better query evaluation performance over the baseline can be achieved.

关键词： SPARQL query RDF data partitioning parallel database

来源：评论

学校读者我要写书评

暂无评论

Design of Intelligent System for Large Scale database

Design of Intelligent System for Large Scale Database

引用

4th International Conference on Intelligent Systems Design and Engineering Applications (ISDEA)

作者： Ye, Zhu Chengdu Univ Informat Technol Chengdu 610225 Sichuan Peoples R China

ISBN: (纸本)9781479927913

In recent years, the amount of mass data has been increased so much that a high performance distributive database to compute the price would be high while guaranteeing the robustness and scalability. To solve this problem, this paper designs an intelligent system for large scale database processing-oriented parallel database engine. Distribution and parallel computing in this system are taken into account. The system framework will be as the communication layer to schedule and coordinate cluster node computing and communications. According to simulation and experiments results, it is obvious that the system performs well and has better scalability and robustness.

关键词： mass data scalability fault tolerance map reduce framework parallel database

来源：评论

学校读者我要写书评

暂无评论

A New parallel Data Cube Construction Scheme

引用

INTERNATIONAL JOURNAL OF GRID AND HIGH PERFORMANCE COMPUTING 2012年第2期4卷 32-45页

作者： Jin, Dong Tsuji, Tatsuo Qingdao Univ Informat Engn Coll Qingdao Peoples R China Univ Fukui Fac Engn Informat Sci Dept Fukui Japan

The pre-computation of data cubes is critical for improving the response time of OLAP (On-Line Analytical Processing) systems. To meet the need for improved performance created by growing data sizes, parallel solutions for data cube construction are becoming increasingly important. This paper presents a new parallel data cube construction scheme based on an extendible multidimensional array, which is dynamically extendible along any dimension without relocating any existing data. The authors have implemented and evaluated their parallel data cube construction methods on shared-memory multiprocessors. Given the performance limit, the methods achieve close to linear speedup with load balance. The authors' experiments also indicate that their parallel methods can be more scalable on higher dimensional data cube construction.

关键词： Data Cube Extendible Array MOLAP parallel database Shared-Memory Multiprocessors

来源：评论

学校读者我要写书评

暂无评论

Managing Skew in the parallel Evaluation of User-Defined Operations

Managing Skew in the Parallel Evaluation of User-Defined Ope...

引用

作者： Kwon, YongChul University of Washington

学位级别：Ph.D.

Science and business are generating data at an unprecedented scale and rate due to ever evolving technologies in computing and sensors. Analyzing big data has become a key skill driving business and science. The challenges in big-data analysis stem not only from the data volume, but also from the diversity of data types to analyze (e.g., text, image, audio, video, and graph) and the various analyses beyond relational algebra that need to be performed (e.g., machine learning, natural language processing, image processing, and graph analysis). The user-defined operation (UDO) is a powerful mechanism to implement complex data processing tasks without changing the core of the parallel data processing engine. Although users can rapidly develop a new data analysis task with UDOs and execute the task in a cluster of computers, achieving high performance is important for users, especially those who do not have an extensive background in programming. This thesis focuses on addressing skew in parallel UDO evaluation. Skew is a problem when there exists a significant variance in the execution time of parallel tasks. In the presence of skew, the benefit of using a parallel system diminishes. Our detailed case study demonstrates that a new data analysis task can be rapidly implemented in a MapReduce-like system, but such implementation may be prone to skew problem during execution. A skew-resilient implementation is possible but requires significant implementation effort and expertise in programming. We also analyze the skew problem in three real workloads and show that skew problem is frequent (more than 40% of long running jobs experience skew). The thesis proposes two techniques to manage skew in parallel UDO evaluations: SkewReduce and SkewTune. SkewReduce is a static data partition optimization technique for feature-extracting applications that are common in scientific analysis. SkewReduce can improve the application runtime by up to 8x compared with a default MapReduce dat

关键词： database load balancing MapReduce parallel database skew udo Computer science

来源：评论

学校读者我要写书评

暂无评论

A high-performance spatial database based approach for pathology imaging algorithm evaluation

引用

Journal of pathology informatics 2013年第1期4卷 5页

作者： Fusheng Wang Jun Kong Jingjing Gao Lee A D Cooper Tahsin Kurc Zhengwen Zhou David Adler Cristobal Vergara-Niedermayr Bryan Katigbak Daniel J Brat Joel H Saltz Department of Biomedical Informatics Emory University USA Center for Comprehensive Informatics Emory University USA.

BACKGROUND:Algorithm evaluation provides a means to characterize variability across image analysis algorithms, validate algorithms by comparison with human annotations, combine results from multiple algorithms for performance improvement, and facilitate algorithm sensitivity studies. The sizes of images and image analysis results in pathology image analysis pose significant challenges in algorithm evaluation. We present an efficient parallel spatial database approach to model, normalize, manage, and query large volumes of analytical image result data. This provides an efficient platform for algorithm evaluation. Our experiments with a set of brain tumor images demonstrate the application, scalability, and effectiveness of the platform. CONTEXT:The paper describes an approach and platform for evaluation of pathology image analysis algorithms. The platform facilitates algorithm evaluation through a high-performance database built on the Pathology Analytic Imaging Standards (PAIS) data model. AIMS:(1) Develop a framework to support algorithm evaluation by modeling and managing analytical results and human annotations from pathology images; (2) Create a robust data normalization tool for converting, validating, and fixing spatial data from algorithm or human annotations; (3) Develop a set of queries to support data sampling and result comparisons; (4) Achieve high performance computation capacity via a parallel data management infrastructure, parallel data loading and spatial indexing optimizations in this infrastructure. MATERIALS AND METHODS:WE HAVE CONSIDERED TWO SCENARIOS FOR ALGORITHM EVALUATION: (1) algorithm comparison where multiple result sets from different methods are compared and consolidated; and (2) algorithm validation where algorithm results are compared with human annotations. We have developed a spatial normalization toolkit to validate and normalize spatial boundaries produced by image analysis algorithms or human annotations. The validated data were

关键词： Algorithm validation parallel database pathology imaging spatial database

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：