In recent years, key-value stores (KV stores) [1]-[3] begin to gain popularity as storage engines for large-scale data applications. KV stores are fundamentally different from traditional SQL databases and with the ke...
详细信息
Writing correct and efficient parallel programs is hard. A lack of overview leads to errors in control- and dataflow, e.g., race conditions, which are hard to find due to their nondeterministic nature. In this paper, ...
详细信息
ISBN:
(纸本)9781538649756
Writing correct and efficient parallel programs is hard. A lack of overview leads to errors in control- and dataflow, e.g., race conditions, which are hard to find due to their nondeterministic nature. In this paper, we present a graphical programming model for parallel stream processingapplications, which improves the overview by visualizing high level dataflow together with explicit and concise annotations for concurrency-related dependency information. The key idea of our approach is twofold: First, we present a powerful graphical task editor together with annotations that enable the designer to define stream properties, task dependencies, and routing information. These annotations facilitate fine-granular and correct parallelization. Second, we propose seamless integration with the safe parallel programming language Rust by providing automated code structure generation from the graphical representation, design patterns for common parallel programming constructs like filters, and a scheduling and runtime environment. We demonstrate the applicability of our approach with a network-based processing system as it is typically found in advanced firewalls.
Image coding is imperative for successful implementation of visual communications applications. With the emergence of enhanced multimedia technology, numerous video applications have emerged like multimedia conferenci...
详细信息
Image coding is imperative for successful implementation of visual communications applications. With the emergence of enhanced multimedia technology, numerous video applications have emerged like multimedia conferencing, video on demand and DVDs. Coding is an essential component of all video applications and it becomes necessary to have improved coding techniques for faster applications. This paper discusses how parallel video coding on load balanced multiprocessor systems can help in incorporating efficient coding techniques like vector quantization into practical applications. Two parallelprocessing platforms will be discussed namely the heterogeneous network of workstations and the TI C40 DSP Chips. The software platforms used for these are the parallel Virtual Machines (PVM) programming model and parallel C respectively. An integration of the two programming models by using a PVM to parallel C Translation and the effect of load balancing for improved performance will also be discussed.
Response time is a key factor of any e-Commerce application, and a set of solutions have been proposed to provide low response time despite network congestions or failures. Being them mostly based on caching of Web ob...
详细信息
In this paper, we propose the design of a library environment, called PARUL (parallel User Library), for distributed memory multiprocessor systems. An important feature of the environment is that it allows the data di...
详细信息
ISBN:
(纸本)0818626720
In this paper, we propose the design of a library environment, called PARUL (parallel User Library), for distributed memory multiprocessor systems. An important feature of the environment is that it allows the data distributed for use of a library function as well as the results generated by the function to be retained in the network of processors to be used by subsequent library functions. The user of the library is given full control over the set of variables that are retained in the network. We describe the implementation details of PARUL on a multi-transputer system and discuss its performance.
In data parallelapplications, a major source of load imbalance is in the uneven distribution of data between the nodes. The major contribution of this paper is the analysis of a new distributed load balancing scheme ...
Due to the rising demand for large-scale data processing, there is a growing interest in both batch processing, where large volumes of data are processed offline, and stream processing, where large quantities of strea...
详细信息
ISBN:
(纸本)9781509028252
Due to the rising demand for large-scale data processing, there is a growing interest in both batch processing, where large volumes of data are processed offline, and stream processing, where large quantities of streaming data are processed online. The dichotomy between these vastly different computing paradigms has led to the development of substantially different methodologies and systems. As there is an increasing number of applications requiring stream and batch processing, there is a need to bridge this gap and offer support for both paradigms. We introduce a new direction for the unification of stream and batch processing, which, contrary to other proposed approaches, uses a stream processing platform as its foundation and supports batch processing on top. Our proof-of-concept implementation of such a middleware layer, called Cyclone, offers the widely popular MapReduce programming model and translates MapReduce jobs for execution on the underlying streaming platform. Cyclone not only achieves a tight integration of batch and stream processing, our evaluation further shows significant performance gains, in particular for sequential and iterative jobs, which naturally arise in many applications.
Pairwise Sequence Alignment is a basic operation in Bioinformatics that is performed thousands of times, in a daily basis. The exact methods proposed in the literature have quadratic time complexity. For this reason, ...
详细信息
Processors with Hyper-Threading technology can improve the performance of applications by permitting a single processor to process data as if it were two processors by executing instructions from different threads in ...
详细信息
Large-scale graph analytics has gained attention during the past few years. As the world is going to be more connected by appearance of new technologies and applications such as social networks, Web portals, mobile de...
详细信息
ISBN:
(纸本)9781509024537
Large-scale graph analytics has gained attention during the past few years. As the world is going to be more connected by appearance of new technologies and applications such as social networks, Web portals, mobile devices, Internet of things, etc, a huge amount of data are created and stored every day in the form of graphs consisting of billions of vertices and edges. Many graph processing frameworks have been developed to process these large graphs since Google introduced its graph processing framework called Pregel in 2010. On the other hand, cloud computing which is a new paradigm of computing that overcomes restrictions of traditional problems in computing by enabling some novel technological and economical solutions such as distributed computing, elasticity and pay-as-you-go models has improved service delivery features. In this paper, we present iGiraph, a cost-efficient Pregel-like graph processing framework for processing large-scale graphs on public clouds. iGiraph uses a new dynamic re-partitioning approach based on messaging pattern to minimize the cost of resource utilization on public clouds. We also present the experimental results on the performance and cost effects of our method and compare them with basic Giraph framework. Our results validate that iGiraph remarkably decreases the cost and improves the performance by scaling the number of workers dynamically.
暂无评论