We present a novel optimization called Last parallel Call Optimization (LPCO) for parallelsystems. The last parallel call optimization can be regarded as a parallel extension of last call optimization found in sequen...
详细信息
ISBN:
(纸本)0818672552
We present a novel optimization called Last parallel Call Optimization (LPCO) for parallelsystems. The last parallel call optimization can be regarded as a parallel extension of last call optimization found in sequential systems. While the LPCO is fairly general, we use and-parallel logic programming systems to illustrate it and to report its performance on multiprocessor systems. The last parallel call optimization leads to improved time and space performance for a majority of and-parallel programs. We also present a generalization of the Last parallel Call Optimization called Nested parallel Call Optimization (NPCO). A major advantage of LPCO and NPCO is that parallelsystems designed for exploiting control parallelism can automatically exploit data parallelism efficiently.
Modern time series databases come with a tag-based query interface that allows users to select time series, which are essentially sequences of timestamped data values, based on a set of specific tags. A tagging index ...
详细信息
ISBN:
(纸本)9781665481069
Modern time series databases come with a tag-based query interface that allows users to select time series, which are essentially sequences of timestamped data values, based on a set of specific tags. A tagging index is an important component that can efficiently provide such tag-based services. However, existing methods store tag information in external databases or time-partitioned data structures, which has a negative impact on query performance. In this paper, we present a novel abstraction for efficient queries of tag information in time series databases: a hybrid tagging index that manages all tags in one place. By managing tag information globally in a single disk-based data structure, we can fundamentally relieve memory pressure and eliminate I/O overhead of duplicate metadata from existing methods. Furthermore, the tagging index is internally partitioned by time to support time range based queries and data retention which are essential to time series databases. We implement the proposed tagging index as a standalone module which can be integrated with time series storage engines. Experiments on the TSBS benchmark show our proposed method can significantly speed up queries by on average 84.0% and 87.2% compared to Prometheus (using a time-partitioned segment method) and Graphite (using an external database for tag management), respectively.
We propose fast probabilistic algorithms with low (i.e., sublinear in the input size) communication volume to check the correctness of operations in Big Data processing frameworks and distributeddatabases. Our checke...
详细信息
ISBN:
(纸本)9781538643686
We propose fast probabilistic algorithms with low (i.e., sublinear in the input size) communication volume to check the correctness of operations in Big Data processing frameworks and distributeddatabases. Our checkers cover many of the commonly used operations, including sum, average, median, and minimum aggregation, as well as sorting, union, merge, and zip. An experimental evaluation of our implementation in Thrill (Bingmann et al., 2016) confirms the low overhead and high failure detection rate predicted by theoretical analysis.
Semijoin has traditionally been relied upon for reducing the communication cost required for distributed query processing. However, judiciously applying join operations as reducers can lead to further reduction in the...
详细信息
ISBN:
(纸本)0818620528
Semijoin has traditionally been relied upon for reducing the communication cost required for distributed query processing. However, judiciously applying join operations as reducers can lead to further reduction in the communication cost. In view of this fact, we explore in this paper the approach of using join operations, in addition to semijoins, as reducers in distributed query processing. We first show that the problem of determining a sequence of join operations for a query graph can be transformed to that of finding a set of cuts to that graph, where a cut to a graph is a partition of the nodes in that graph. In light of the mapping we develop an efficient heuristic algorithm to determine an effective sequence of join reducers for a query. The algorithm using the concept of divide-and-conquer is shown to have polynomial time complexity. Examples are also given to illustrate our results.
This paper focuses on parallel query optimization. We consider the operator problem and introduce a new class of execution strategies called Linear-oriented Bushy Trees (LBT). Compared to the related approach of the G...
详细信息
This paper focuses on parallel query optimization. We consider the operator problem and introduce a new class of execution strategies called Linear-oriented Bushy Trees (LBT). Compared to the related approach of the General Bushy Trees (GBT) a significant complexity reduction of the operator ordering problem can be derived theoretically and demonstrated experimentally (e.g, compared with GBTs, LBTs authorize optimization time improvement that can reach up to 49%) without losing quality. Finally we demonstrate that existing commercial parallel query optimizers need little extension mod ifications in order to handle LBTs. (C) 2000 Elsevier Science B.V. All rights reserved.
Emerging applications in areas such as bioinformatics, data analytics, semantic databases and knowledge discovery employ datasets from tens to hundreds of terabytes. Currently, only distributed memory clusters have en...
详细信息
ISBN:
(纸本)9780769552071
Emerging applications in areas such as bioinformatics, data analytics, semantic databases and knowledge discovery employ datasets from tens to hundreds of terabytes. Currently, only distributed memory clusters have enough aggregate space to enable in-memory processing of datasets of this size. However, in addition to large sizes, the data structures used by these new application classes are usually characterized by unpredictable and fine-grained accesses: i.e., they present an irregular behavior. Traditional commodity clusters, instead, exploit cache-based processor and high-bandwidth networks optimized for locality, regular computation and bulk communication. For these reasons, irregular applications are inefficient on these systems, and require custom, hand-coded optimizations to provide scaling in both performance and size. Lightweight software multithreading, which enables tolerating data access latencies by overlapping network communication with computation, and aggregation, which allows reducing overheads and increasing bandwidth utilization by coalescing fine-grained network messages, are key techniques that can speed up the performance of large scale irregular applications on commodity clusters. In this paper we describe GMT (Global Memory and Threading), a runtime system library that couples software multithreading and message aggregation together with a Partitioned Global Address Space (PGAS) data model to enable higher performance and scaling of irregular applications on multi-node systems. We present the architecture of the runtime, explaining how it is designed around these two critical techniques. We show that irregular applications written using our runtime can outperform, even by orders of magnitude, the corresponding applications written using other programming models that do not exploit these techniques.
This paper makes two contributions. First, we introduce a model for evaluating the performance of data allocation and replication algorithms in distributeddatabases. The model is comprehensive in the sense that it ac...
详细信息
This paper makes two contributions. First, we introduce a model for evaluating the performance of data allocation and replication algorithms in distributeddatabases. The model is comprehensive in the sense that it accounts for I/O cost, for communication cost, and, because of reliability considerations, for limits on the minimum number of copies of the object. The model captures existing replica-management algorithms, such as read-one-write-all, quorum-consensus, etc. These algorithms are static in the sense that, in the absence of failures,, the copies of each object are allocated to a fixed set of processors. In modern distributeddatabases, particularly in mobile computing environments, processors will dynamically store objects in their local database and will relinquish them. Therefore, as a second contribution of this paper, we introduce an algorithm for automatic dynamic allocation of replicas to processors. Then, using the new model, we compare the performance of the traditional read-one-write-all static allocation algorithm to the performance of the dynamic allocation algorithm. As a result, we obtain the relationship between the communication cost and I/O cost for which static allocation is superior to dynamic allocation, and the relationships for which dynamic allocation is superior.
The models and languages most widely used for distributedsystems programming (such as ADA, CSP, and OCCAM) do not explicitly consider the process execution site, they assume a synchronous communication scheme of “re...
详细信息
ISBN:
(纸本)0818620528
The models and languages most widely used for distributedsystems programming (such as ADA, CSP, and OCCAM) do not explicitly consider the process execution site, they assume a synchronous communication scheme of “rendezvous” style, and require that each process recognize the names of their interlocutors. JOYCE+ is a modification of the JOYCE language defined by Brinch Hansen [Brinch 87, Brinch 89]. In the JOYCE+ model each process is executed in a specific site, it is assumed that the communication between processes at the same site is much more faster than between processes at different sites, and it is also assumed that the communication between processes is not always reliable. For this multi-site environment an asynchronous communication scheme is considered, eliminating waiting states for a process that sends a message to another process. Furthermore, modelling communication between processes through “channel” objects, the spaces for process names become *** this paper the JOYCE+ model for multi-site distributedsystems is presented and the operational semantic of the asynchronous communication between processes is illustrated with Petri Nets. The syntax of the JOYCE+ language is presented in terms of the Guarded Commands language [Dijkstra 75]. The expressive power of the JOYCE+ language in distributed synchronization problems with timeout handling is illustrated through examples. Finally, we discuss about software development environments (based on JOYCE+) for distributedsystems over multi- and mono-process computer *** Terms - Languages for distributedsystems, Tools and methodologies for distributedsystems.
This article describes the keynote speech on INODE presented at Fourth international Workshop on systems and Network Telemetry and Analytics (SNTA) which is collocated with international ACM symposium on High -Perform...
详细信息
ISBN:
(纸本)9781450383868
This article describes the keynote speech on INODE presented at Fourth international Workshop on systems and Network Telemetry and Analytics (SNTA) which is collocated with international ACM symposium on High -Performance parallel and distributed Computing (HPDC) on June 21 in Stockholm, Sweden.
Using simulation and probabilistic analysis, the authors study the performance of an algorithm to read entire databases with locking concurrency control allowing multiple readers or an exclusive writer. The algorithm ...
详细信息
ISBN:
(纸本)0818608935
Using simulation and probabilistic analysis, the authors study the performance of an algorithm to read entire databases with locking concurrency control allowing multiple readers or an exclusive writer. The algorithm runs concurrently with the normal transaction processing (on-the-fly) and locks the entities in the database one by one (incremental). The analysis compares different strategies to resolve the conflicts between the global read algorithm and update. Since the algorithm is parallel in nature, its interference with normal transactions is minimized in parallel and distributeddatabases. A simulation study shows that one variant of the algorithm can read the entire database with very little overhead and interference with the updates.
暂无评论