The vertex-centric programming model, known as "think like a vertex", is being used more and more to support various big graph processing methods through iterative super steps that execute in parallel a user...
详细信息
ISBN:
(纸本)9781450342193
The vertex-centric programming model, known as "think like a vertex", is being used more and more to support various big graph processing methods through iterative super steps that execute in parallel a user-defined vertex program over each vertex of a graph. However, the imperative and message-passing style of existing systems makes defining a vertex program unintuitive. In this paper, we show that one can benefit more from "Thinking like a vertex" by "Behaving like a function" rather than "Acting like a procedure" with full use of side effects and explicit control of message passing, state, and termination. We propose a functional approach to vertex-centric graph processing in which the computation at every vertex is abstracted as a higher-order function and present Fregel, a new domain-specific language. Fregel has clear functional semantics, supports declarative description of vertex computation, and can be automatically translated into Pregel, an emerging imperative-style distributed graph processing framework, and thereby achieve promising performance. Experimental results for several typical examples show the promise of this functional approach.
The vertex-centric programming model, known as "think like a vertex", is being used more and more to support various big graph processing methods through iterative super steps that execute in parallel a user...
详细信息
The vertex-centric programming model, known as "think like a vertex", is being used more and more to support various big graph processing methods through iterative super steps that execute in parallel a user-defined vertex program over each vertex of a graph. However, the imperative and message-passing style of existing systems makes defining a vertex program unintuitive. In this paper, we show that one can benefit more from "Thinking like a vertex" by "Behaving like a function" rather than "Acting like a procedure" with full use of side effects and explicit control of message passing, state, and termination. We propose a functional approach to vertex-centric graph processing in which the computation at every vertex is abstracted as a higher-order function and present Fregel, a new domain-specific language. Fregel has clear functional semantics, supports declarative description of vertex computation, and can be automatically translated into Pregel, an emerging imperative-style distributed graph processing framework, and thereby achieve promising performance. Experimental results for several typical examples show the promise of this functional approach.
We consider here reduction of send latencies for send-and-forget type communications, where senders do not reuse data that have already been sent. We introduce a commit based message sending approach, and describe the...
详细信息
We consider here reduction of send latencies for send-and-forget type communications, where senders do not reuse data that have already been sent. We introduce a commit based message sending approach, and describe the corresponding application programming interfaces. These interfaces allow applications to take advantage of message buffering, such as overlapping of computations and communications, while avoiding performance degradation due to intermediate memory copy. The approach works by allowing applications to safely access the implementation buffer through the provided interfaces. Experimental results show that the new approach is effective, and reduces the interface latency significantly compared to related approaches. Since send-and-forget type communications are often observed in skeletalprogramming, we demonstrate the approach by implementing a pipeline algorithmic skeleton.
Computations on two-dimensional arrays such as matrices and images are one of the most fundamental and ubiquitous things in computational science and its vast application areas, but development of efficient parallel p...
详细信息
Computations on two-dimensional arrays such as matrices and images are one of the most fundamental and ubiquitous things in computational science and its vast application areas, but development of efficient parallel programs on two-dimensional arrays is known to be hard. In this paper, we propose a compositional framework that supports users, even with little knowledge about parallel machines, to develop both correct and efficient parallel programs on dense two-dimensional arrays systematically. The key feature of our framework is a novel use of the abide-tree representation of two-dimensional arrays. The presentation not only inherits the advantages of tree representations of matrices where recursive blocked algorithms can be defined to achieve better performance, but also supports transformational development of parallel programs and architecture-independent implementation owing to its solid theoretical foundation - the theory of constructive algorithmics.
skeletal parallel programming enables programmers to build a parallel program from ready-made components (parallel primitives) for which efficient implementations are known to exist, making both the parallel program d...
详细信息
skeletal parallel programming enables programmers to build a parallel program from ready-made components (parallel primitives) for which efficient implementations are known to exist, making both the parallel program development and the parallelization process easier. Constructing efficient parallel programs is often difficult, however, due to difficulties in selecting a proper combination of parallel primitives and in implementing this combination Without having unnecessary creations and exchanges of data among parallel primitives and processors. To overcome these difficulties, we propose a powerful and general parallel skeleton, accumulate, which can be used to naturally code efficient Solutions to problems as well as be efficiently implemented in parallel using Message Passing Interface (MPI).
暂无评论