检索结果-内蒙古大学图书馆

9th parallel Data Storage Workshop (PDSW)

作者： Chasapis, Konstantinos Dolz, Manuel F. Kuhn, Michael Ludwig, Thomas Univ Hamburg Dept Informat Bundesstr 45a D-20146 Hamburg Germany

ISBN: (纸本)9781479970254

With the emergence of multi-core and multi-socket non-uniform memory access (NUMA) platforms in recent years, new software challenges have arisen to use them efficiently. In the field of high performance computing (HPC), parallel programming has always been the key factor to improve applications performance. However, the implications of parallel architectures in the system software has been overlooked until recently. In this work, we examine the implications of such platforms in the performance scalability of the Lustre parallel distributed file system's metadata server (MDS). We run our experiments on a four socket NUMA platform that has 48 cores. We leverage the mdtest benchmark to generate appropriate metadata workloads and include configurations with varying numbers of active cores and mount points. Additionally, we compare Lustre's metadata scalability with the local file systems ext4 and XFS. The results demonstrate that Lustre's metadata performance is limited to a single socket and decreases when more sockets are used. We also observe that the MDS's back-end device is not a limiting factor regarding the performance.

关键词： meta data multiprocessing systems network operating systems parallel databases storage management Lustre parallel distributed file system metadata server MDS back-end device XFS ext4 four socket NUMA platform local file systems mdtest benchmark metadata workloads multisocket platform nonuniform memory access performance scalability Kernel Linux Performance evaluation Scalability Servers Sockets Metadata Non-Uniform Memory Access Socket Network operating systems Power outlets Exfoliation Syndrome Servers parallel databases Multiprocessing systems Scalability Store management Linux Performance Evaluation Systems software Platform

来源：评论

学校读者我要写书评

暂无评论

SciDB: A Database Management System for Applications with Complex Analytics

引用

COMPUTING IN SCIENCE & ENGINEERING 2013年第3期15卷 54-62页

作者： Stonebraker, Michael Brown, Paul Zhang, Donghui Becla, Jacek MIT Comp Sci & Artificial Intelligence Lab Cambridge MA 02139 USA MIT Intel Sci & Technol Ctr Cambridge MA 02139 USA

A description and discussion of the SciDB database management system focuses on lessons learned, application areas, performance comparisons against other solutions, and additional approaches to managing data and compl... 详细信息

关键词： array databases complex analytics large-scale matrix operations parallel databases scientific computing scientific databases Scientific computing Scientific database database management systems parallel databases

来源：评论

学校读者我要写书评

暂无评论

Cogset: a high performance MapReduce engine

引用

CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE 2013年第1期25卷 2-23页

作者： Valvag, S. V. Johansen, Dag Kvalnes, Age Univ Tromso Dept Comp Sci Tromso Norway

Cogset is a generic and efficient engine for reliable storage and parallel processing of distributed data sets. It supports a number of high-level programming interfaces, including a MapReduce interface compatible with Hadoop. In this paper, we present Cogset's architecture and evaluate its performance as a MapReduce engine, comparing it with Hadoop. Our results show that Cogset generally outperforms Hadoop by a significant margin. We investigate the underlying causes of this difference in performance and demonstrate some relatively minor modifications that markedly improve Hadoop's performance, closing some of the gap. Copyright (c) 2012 John Wiley & Sons, Ltd.

关键词： MapReduce parallel databases Hadoop benchmark

来源：评论

学校读者我要写书评

暂无评论

A Selective Checkpointing Mechanism for Query Plans in a parallel Database System

A Selective Checkpointing Mechanism for Query Plans in a Par...

引用

IEEE International Conference on Big Data (Big Data)

作者： Chen, Ting Taura, Kenjiro Univ Tokyo Tokyo 1138654 Japan

ISBN: (纸本)9781479912926;9781479912933

Most existing parallel database systems achieve fault tolerance by aborting unfinished queries upon a failure and restart the entire from the beginning. This is inefficient for long running queries of OLAP workloads. To solve this problem, this paper presents a selective checkpointing mechanism which materializes the outputs of some necessary operators, enabling to resume queries from middle of the execution upon failures. Each query is represented by a DAG of relational operators in which data are typically pipelined between operators. The goal of the mechanism is to find a set of operators whose outputs are worth being checkpointed to minimize the expected runtime of the whole query. It firstly provides a cost model to estimate the expected runtime of a whole query plan under a given failure probability for each operator. Then a divide-and-conquer algorithm is proposed to find a close-to-optimal solution to the problem. The algorithm divides the query plan into sub-plans with smaller search spaces. For a given query plan with n operators, the algorithm runs in O(n) time. The mechanism is implemented in a shared-nothing parallel database system called ParaLite which provides a coordination layer to glue many SQLite instances together, and parallelizes SQL queries across them. The experimental results indicate that different fault-tolerant strategies affect the overall runtimes of queries. Our selective checkpointing mechanism can choose reasonable operators to be checkpointed and outperforms other fault-tolerant strategies. In addition, the divide-and-conquer algorithm taken by our mechanism has a smaller overhead than brute-force approach while keeping a similar effectiveness.

关键词： SQL checkpointing data mining directed graphs divide and conquer methods fault tolerance parallel databases probability query processing

来源：评论

学校读者我要写书评

暂无评论

Accelerating Semantic Graph databases on Commodity Clusters

Accelerating Semantic Graph Databases on Commodity Clusters

引用

IEEE International Conference on Big Data (Big Data)

作者： Morari, Alessandro Castellana, Vito Giovanni Haglin, David Feo, John Weaver, Jesse Tumeo, Antonino Villa, Oreste Pacific NW Natl Lab Richland WA 99352 USA NVIDIA Santa Clara CA USA

ISBN: (纸本)9781479912926;9781479912933

We are developing a full software system for accelerating semantic graph databases on commodity cluster that scales to hundreds of nodes while maintaining constant query throughput. Our framework comprises a SPARQL to C++ compiler, a library of parallel graph methods and a custom multithreaded runtime layer, which provides a Partitioned Global Address Space (PGAS) programming model with fork/join parallelism and automatic load balancing over a commodity clusters. We present preliminary results for the compiler and for the runtime.

关键词： C++ language graph theory multi-threading parallel databases program compilers query processing resource allocation

来源：评论

学校读者我要写书评

暂无评论

Optimizations on the parallel Virtual File System Implementation Integrated with Object-Based Storage Devices

Optimizations on the Parallel Virtual File System Implementa...

引用

15th IEEE International Conference on Cluster Computing (CLUSTER)

作者： Karakoyunlu, Cengiz Chandy, John A. Univ Connecticut Dept Elect & Comp Engn Storrs CT 06269 USA

ISBN: (纸本)9781479908981

Object-Based Storage Devices (OSDs) offer an object-based data layout instead of using the traditional block-based design. Object storage ideas are increasingly being used in parallel file systems and cloud storage. Previous work has implemented the parallel Virtual File System (PVFS) on OSDs by moving the metadata and data operations to the OSDs. hi this paper, we present two new methods to further improve the performance of the existing PVFS-OSD implementation. These methods are using object collections to represent traditional directories on OSDs and delaying the creation of an OSD object until it is accessed by a write operation. The optimization methods are not particular to PVFS-OSD implementation and could be used with other parallel file systems-object storage implementations too. Experimental evaluations show that using collections yields up to 29 times of improvement in throughput;whereas delaying the creation of OSD objects provides 25% throughput improvement.

关键词： meta data optimisation parallel databases

来源：评论

学校读者我要写书评

暂无评论

Workload Management for Big Data Analytics

Workload Management for Big Data Analytics

引用

29th IEEE International Conference on Data Engineering (ICDE)

作者： Aboulnaga, Ashraf Babu, Shivnath Univ Waterloo Cheriton Sch Comp Sci Waterloo ON N2L 3G1 Canada Duke Univ Dept Comp Sci Durham NC 27706 USA

ISBN: (纸本)9781467349093;9781467349086

parallel database systems and MapReduce systems (most notably Hadoop) are essential components of today's infrastructure for Big Data analytics. These systems process multiple concurrent workloads consisting of complex user requests, where each request is associated with an (explicit or implicit) service level objective. For example, the workload of a particular user or application may have a higher priority than other workloads. Or a particular workload may have strict deadlines for the completion of its requests.

关键词： Workload Database systems parallel databases big data Explicit Completion

来源：评论

学校读者我要写书评

暂无评论

parallel Pipelined Filter Ordering with Precedence Constraints

引用

ACM TRANSACTIONS ON ALGORITHMS 2012年第4期8卷 41-41页

作者： Deshpande, Amol Hellerstein, Lisa Univ Maryland College Pk MD 20742 USA NYU Polytech Inst Brooklyn NY USA

In the parallel pipelined filter ordering problem, we are given a set of n filters that run in parallel. The filters need to be applied to a stream of elements, to determine which elements pass all filters. Each filter has a rate limit r(i) on the number of elements it can process per unit time, and a selectivity p(i), which is the probability that a random element will pass the filter. The goal is to maximize throughput. This problem appears naturally in a variety of settings, including parallel query optimization in databases and query processing over Web services. We present an O(n(3)) algorithm for this problem, given tree-structured precedence constraints on the filters. This extends work of Condon et al. [2009] and Kodialam [2001], who presented algorithms for solving the problem without precedence constraints. Our algorithm is combinatorial and produces a sparse solution. Motivated by join operators in database queries, we also give algorithms for versions of the problem in which "filter" selectivities may be greater than or equal to 1. We prove a strong connection between the more classical problem of minimizing total work in sequential filter ordering (A), and the parallel pipelined filter ordering problem (B). More precisely, we prove that A is solvable in polynomial time for a given class of precedence constraints if and only if B is as well. This equivalence allows us to show that B is NP-Hard in the presence of arbitrary precedence constraints (since A is known to be NP-Hard in that setting).

关键词： Pipelined filter ordering parallel databases query optimization flow algorithms sequential testing

来源：评论

学校读者我要写书评

暂无评论

Dynamic Prioritization of Database Queries

Dynamic Prioritization of Database Queries

引用

IEEE 27th International Conference on Data Engineering (ICDE 2011)

作者： Narayanan, Sivaramakrishnan Waas, Florian Greenplum Inc EMC Corp San Mateo CA 94403 USA

ISBN: (纸本)9781424489589

Enterprise database systems handle a variety of diverse query workloads that are of different importance to the business. For example, periodic reporting queries are usually mission critical whereas ad-hoc queries by analysts tend to be less crucial. It is desirable to enable database administrators to express (and modify) the importance of queries at a simple and intuitive level. The mechanism used to enforce these priorities must be robust, adaptive and efficient. In this paper, we present a mechanism that continuously determines and re-computes the ideal target velocity of concurrent database processes based on their run-time statistics to achieve this prioritization. In this scheme, every process autonomously adjusts its resource consumption using basic control theory principles. The self-regulating and decentralized design of the system enables effective prioritization even in the presence of exceptional situations, including software defects or unexpected/unplanned query termination with no measurable overhead. We have implemented this approach in Greenplum parallel Database and demonstrate its effectiveness and general applicability in a series of experiments.

关键词： Computer architecture Equations Greenplum parallel database Mathematical model Monitoring Query processing business data processing concurrent database processes control theory principles database queries dynamic prioritization enterprise database systems parallel databases query processing resource consumption run-time statistics

来源：评论

学校读者我要写书评

暂无评论

Embarrassingly Scalable Database Systems

Embarrassingly Scalable Database Systems

引用

IEEE 27th International Conference on Data Engineering (ICDE 2011)

作者： Ailamaki, Anastasia Ecole Polytech Fed Lausanne Sch Comp & Commun Sci Data Intens Applicat & Syst DIAS Lab Stn 14 CH-1015 Lausanne Switzerland

From Wikipedia-An embarrassingly parallel workload (or embarrassingly parallel problem) is one for which little or no effort is required to separate the problem into a number of parallel tasks. This is often the case ... 详细信息

ISBN: (纸本)9781424489589

关键词： Database systems Business intelligence Scalability database machines PROCESSOR Database Hosts Processor Performance parallel databases Computer chips DNA Microarrays

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：