In this paper, we present a new parallel method named SDFEM that enables frequent pattern mining (FPM) on cluster with multiple multi-core compute nodes to provide high performance. SDFEM is distinguished from previou...
详细信息
LHCbDirac is a Dirac extension used for all the distributed computing activities of LHCb. Dirac has proven to be a scalable, and extensible software. LHCb is, up to now, its main user and main contributor, and LHCb cr...
详细信息
One way to efficiently utilize the coming exascale machines is to support a mixture of applications in various domains, such as traditional large-scale HPC, the ensemble runs, and the fine-grained many-task computing ...
详细信息
The proceedings contain 18 papers. The special focus in this conference is on Static Analysis. The topics include: Static analysis of non-interference in expressive low-level languages;static analysis with set-closure...
ISBN:
(纸本)9783662482872
The proceedings contain 18 papers. The special focus in this conference is on Static Analysis. The topics include: Static analysis of non-interference in expressive low-level languages;static analysis with set-closure in secrecy;a binary decision tree abstract domain functor;precise data flow analysis in the presence of correlated method calls;may-happen-in-parallel analysis for asynchronous programs with inter-procedural synchronization;shape analysis for unstructured sharing;synthesizing heap manipulations via integer linear programming;effective soundness-guided reflection analysis;a type system for javascript with fixed object layout;refinement type inference via horn constraint optimization;a simple abstraction of arrays and maps by program translation;property-based polynomial invariant generation using sums-of-squares optimization;a case study on the correspondence between top-down and bottom-up analysis;parallel cost analysis of distributedsystems;a forward analysis for recurrent sets and unbounded-time analysis of guarded LTI systems with inputs by abstract acceleration.
Data replication is a common technique used for fault-tolerance in reliable distributedsystems. In geo-replicated systems and the cloud, it additionally provides low latency. Recently, causal consistency in such syst...
详细信息
ISBN:
(纸本)9781467376853
Data replication is a common technique used for fault-tolerance in reliable distributedsystems. In geo-replicated systems and the cloud, it additionally provides low latency. Recently, causal consistency in such systems has received much attention. However, all existing works assume the data is fully replicated. This greatly simplifies the design of the algorithms to implement causal consistency. In this paper, we propose that it can be advantageous to have partial replication of data, and we propose two algorithms for achieving causal consistency in such systems where the data is only partially replicated. This is the first work that explores causal consistency for partially replicated geo-replicated systems. We also give a special case algorithm for causal consistency in the full-replication case.
With nodes in Delay Tolerant Network(DTN) distributing sparsely and moving rapidly, they usually suffer from intermittent connections and communications, thus bringing about limited message forwarding opportunities. A...
详细信息
ISBN:
(纸本)9781450337625
With nodes in Delay Tolerant Network(DTN) distributing sparsely and moving rapidly, they usually suffer from intermittent connections and communications, thus bringing about limited message forwarding opportunities. All these could lead to inefficient forwarding, low delivery, long latency and limited transmission capacity in performance. In this paper, efficient encoding and decision methods are presented and integrated into the DTN routing strategy. The custody-encoding-forwarding mode is designed by merging the random linear network coding into the DTN routing, together with the replica re-allocation and memory management, and built on that encoding scheme, the intra/inter flow adaptive collaborative network coding is elaborated to implement the fresh custody-decision-encoding-forwarding mode. A decision-making strategy based on Bayesian Network(BN) measures the "degree" in a specific generation in networks by taking comprehensive consideration of current and historical network conditions to enhance the network robustness and self-adaptivity. By evaluating the delivery, delay and overhead performance on ONE and MATLAB platforms, the effectiveness of proposed strategies is validated in the end.
In this paper, we study parallel data access on distributed file systems, e.g, the Hadoop file system. Our experiments show that parallel data read requests are often served data remotely and in an imbalanced fashion....
详细信息
ISBN:
(纸本)9781479986507
In this paper, we study parallel data access on distributed file systems, e.g, the Hadoop file system. Our experiments show that parallel data read requests are often served data remotely and in an imbalanced fashion. This results in a serious disk access and data transfer contention on certain cluster/storage nodes. We conduct a complete analysis on how remote and imbalanced read patterns occur and how they are affected by the size of the cluster. We then propose a novel method to Optimize parallel Data Access on distributed File systems referred to as Opass. The goal of Opass is to reduce remote parallel data accesses and achieve a higher balance of data read requests between cluster nodes. To achieve this goal, we represent the data read requests that are issued by parallel applications to cluster nodes as a graph data structure where edges weights encode the demands of data locality and load capacity. Then we propose new matching-based algorithms to match processes to data based on the configurations of the graph data structure so as to compute the maximum degree of data locality and balanced access. Our proposed method can benefit parallel data-intensive analysis with various parallel data access strategies. Experiments are conducted on PRObEs Marmot 128-node cluster tested and the results from both benchmark and well-known parallel applications show the performance benefits and scalability of Opass.
Column-store in-memory databases have received a lot of attention because of their fast query processing response times on modern multi-core machines. Among different database operations, group by/aggregate is an impo...
详细信息
ISBN:
(纸本)9781467376853
Column-store in-memory databases have received a lot of attention because of their fast query processing response times on modern multi-core machines. Among different database operations, group by/aggregate is an important and potentially costly operation. Moreover, sort-based and hash-based algorithms are the most common ways of processing group by/aggregate queries. While sort-based algorithms are used in traditional Data Base Management systems (DBMS), hash based algorithms can be applied for faster query processing in new columnar databases. Besides, Graphical Processing Units (GPU) can be utilized as fast, high bandwidth co-processors to improve the query processing performance of columnar databases. The focus of this article is on the prototype for group by/aggregate operations that we created to exploit GPUs. We show different hash based algorithms to improve the performance of group by/aggregate operations on GPU. One of the parameters that affect the performance of the group by/aggregate algorithm is the number of groups and hashing algorithm. We show that we can get up to 7.6x improvement in kernel performance compared to a multi-core CPU implementation when we use a partitioned multi-level hash algorithm using GPU shared and global memories.
Big data and the Internet of Things era continue to challenge computational systems. Several technology solutions such as NoSQL databases have been developed to deal with this challenge. In order to generate meaningfu...
详细信息
ISBN:
(纸本)9781467376853
Big data and the Internet of Things era continue to challenge computational systems. Several technology solutions such as NoSQL databases have been developed to deal with this challenge. In order to generate meaningful results from large datasets, analysts often use a graph representation which provides an intuitive way to work with the data. Graph vertices can represent users and events, and edges can represent the relationship between vertices. Graph algorithms are used to extract meaningful information from these very large graphs. At MIT, the Graphulo initiative is an effort to perform graph algorithms directly in NoSQL databases such as Apache Accumulo or SciDB, which have an inherently sparse data storage scheme. Sparse matrix operations have a history of efficient implementations and the Graph Basic Linear Algebra Subprogram (Graph BLAS) community has developed a set of key kernels that can be used to develop efficient linear algebra operations. However, in order to use the Graph BLAS kernels, it is important that common graph algorithms be recast using the linear algebra building blocks. In this article, we look at common classes of graph algorithms and recast them into linear algebra operations using the Graph BLAS building blocks.
In distributed storage systems like parallel file systems or storage virtualization middleware, data replication is the mainly used solution to implement data avaialability. The more replicas are distributed among nod...
详细信息
In distributed storage systems like parallel file systems or storage virtualization middleware, data replication is the mainly used solution to implement data avaialability. The more replicas are distributed among nodes, the more robust is the storage system. However, the price to pay for this dependability becomes significant, due to both direct costs (the price of disks) and indirect costs (the energy consumption of this large amount of disks needed). In order to lower the disk space needed for a given availalbility, Erasure Resilient Codes (referred to as ERC after this) are of interest and start to be implemented in this context. However, the use of such codes involves some new problems in data management. In fact, if some constraints like data concurrency can be solved using classical ways, others like coherency protocols need some adaptations in order to fit this context. In this paper, we present an adaptation of trapezoid protocol in the context of ERC schemes (instead of full replication). This new quorum protocol shows an increase of storage space efficiency while maintaining a high level of availability for read and writes operations.
暂无评论