In today's world, stream processing systems have become important, as applications like media broadcasting, sensor network monitoring and on-line data analysis increasingly rely on real-time stream processing. In ...
详细信息
In today's world, stream processing systems have become important, as applications like media broadcasting, sensor network monitoring and on-line data analysis increasingly rely on real-time stream processing. In this paper, we propose a distributed stream processing system that composes stream processing applications dynamically, while meeting their rate demands. Our system consists of the following components: (1) a distributed component discovery algorithm that discovers components available at nodes on demand, (2) resource monitoring techniques to maintain current resource availability information, (3) a scheduling algorithm that schedules application execution, and (4) a minimum cost composition algorithm that composes applications dynamically based on component and resource availability and scheduling demands. Our detailed experimental results, over the PlanetLab testbed, demonstrate the performance and efficiency of our approach.
Huge energy consumption has become a critical bottleneck for further applying large-scale cluster systems to build new data centers. Among various components of a data center, storage subsystems are one of the biggest...
详细信息
Huge energy consumption has become a critical bottleneck for further applying large-scale cluster systems to build new data centers. Among various components of a data center, storage subsystems are one of the biggest consumers of energy. In this paper, we propose a novel buffer-disk based framework for large-scale and energy-efficient parallel storage systems. To validate the efficiency of the proposed framework, a buffer-disk scheduling algorithm is designed and implemented. Our algorithm can provide more opportunities for underlying disk power management schemes to save energy by keeping a large number of idle data disks in sleeping mode as long as possible. The trace-driven simulation results based on a revised disksim simulator show that this new framework can significantly improves the energy efficiency of large-scale parallel storage systems.
We present a scalable, hierarchical control system for the dynamic resource management of a distributed real-time embedded (DRE) system. This DRE is inspired by the DARPA adaptive and reflective middleware systems (AR...
详细信息
We present a scalable, hierarchical control system for the dynamic resource management of a distributed real-time embedded (DRE) system. This DRE is inspired by the DARPA adaptive and reflective middleware systems (ARMS) program. The goal of the control system is to simultaneously manage multiple resources and QoS concerns using a utility-driven approach for decision making and performance evaluation. At each level of the control hierarchy there are multiple local controllers which autonomously make decisions to optimize their local utility. The controllers in the hierarchy can use different, localized resource control algorithms and the system's user can tune the operations of the local controllers. We discuss how the selections of local control algorithms affect the behavior of the overall system. The control system is designed to be easily adaptable to other multi-tiered DRE systems.
Scheduling divisible loads in distributedsystems is the subject of divisible load theory (DLT). In this paper we show that coalitional game theory is a natural fit for modeling DLT as the participants in the scheduli...
详细信息
ISBN:
(纸本)9780769529172;0769529178
Scheduling divisible loads in distributedsystems is the subject of divisible load theory (DLT). In this paper we show that coalitional game theory is a natural fit for modeling DLT as the participants in the scheduling algorithm must cooperate in order to execute a job. We devise a coalitional scheduling game in which the job owners and the independent organizations that own processors form coalitions in order to maximize their profits. We examine the payoffs to the participants and show that the core of the proposed coalitional scheduling game is non-empty. Then we examine the "fair sharing" of the payoffs among the participants using the Shapley value. Finally we study by simulation the properties of the proposed coalitional scheduling game considering different distributedsystems configurations.
The problem of counting specified combinations of a given set of variables arises in many statistical and data mining applications. To solve this problem, we introduce the PDtree data structure, which avoids exponenti...
详细信息
The problem of counting specified combinations of a given set of variables arises in many statistical and data mining applications. To solve this problem, we introduce the PDtree data structure, which avoids exponential time and space complexity associated with prior work by allowing user specification of the tree structure. A straightforward parallelization approach using a Cray MTA-2 provides a speedup that is linear in the number of processors, but introduces nondeterminism into probability estimates. We prove a general convergence result that bounds the non-deterministic deviation of probability estimates relative to a sequential implementation. Beyond PDtrees, this convergence result applies to any counting application that takes a multithreaded streaming approach.
Grid computing can be used for high performance computations. However, a serious difficulty in concurrent programming of such heterogeneous systems is how to deal with scheduling and load balancing of such systems whi...
详细信息
Grid computing can be used for high performance computations. However, a serious difficulty in concurrent programming of such heterogeneous systems is how to deal with scheduling and load balancing of such systems which may consist of heterogeneous computers on different sites. distributed scheduling schemes suitable for parallel loops with independent iterations on heterogeneous computer clusters have been proposed and analyzed in the past. In this article, we implement the previous schemes in MPICH-G2 and MPIg on the TeraGrid. We present performance results for three loop scheduling schemes on single and multi-site TeraGrid clusters.
We propose a two-layered approach for exploiting different forms of concurrency in complex systems: we specify computational components in our functional array language SAC, which exploits data parallel properties of ...
详细信息
We propose a two-layered approach for exploiting different forms of concurrency in complex systems: we specify computational components in our functional array language SAC, which exploits data parallel properties of array processing code. The declarative stream processing language S-Net is used to orchestrate the collaborative behaviour of these components in a streaming network. We illustrate our approach by a hybrid implementation of a sudoku puzzle solver as a representative for more complex search problems.
Self-stabilization is a theoretical framework of non-masking fault-tolerant distributed algorithms. A self-stabilizing system tolerates any kind and any finite number of transient faults, such as message loss, memory ...
详细信息
Self-stabilization is a theoretical framework of non-masking fault-tolerant distributed algorithms. A self-stabilizing system tolerates any kind and any finite number of transient faults, such as message loss, memory corruption, and topology change. Because such transient faults occur so frequently in mobile ad hoc networks, distributed algorithms on them should tolerate such events. In this paper, we propose a self-stabilizing distributed approximation algorithm for the minimum connected dominating set, which can be used, for example, as a virtual backbone or routing in mobile ad hoc networks. The size of the solution by our algorithm is at most 8 |D opt | + 1, where Dopt is a minimum connected dominating set. The time complexity is O(n 2 ) steps.
Many performance problems observed in high end systems are actually caused by the runtime system and not the application code. Detecting these cases require parallel performance tools to incorporate information about ...
详细信息
Many performance problems observed in high end systems are actually caused by the runtime system and not the application code. Detecting these cases require parallel performance tools to incorporate information about the runtime system; however many current tools do not. We present a test suite for evaluating the ability of performance tools to reach correct diagnosis in cases where a problem is caused by the runtime environment. We include a set of results for one of the tests, which measures application performance as NFS server load is increased. We also present a model for performance diagnosis that combines system and application level information.
Symbolic computing is one of fastest growing areas of scientific computing. An overview of the state-of-the-art in symbolic computations on distributed architectures, in particular Web and Grid architectures, is prese...
详细信息
ISBN:
(纸本)9780769529172;0769529178
Symbolic computing is one of fastest growing areas of scientific computing. An overview of the state-of-the-art in symbolic computations on distributed architectures, in particular Web and Grid architectures, is presented. The background information, including typical application areas, is followed by a list of past and on-going projects involving symbolic computations on distributed computing environments. To illustrate in more details issues involved in porting computer algebra systems to the Grid, some case studies involving popular environments are presented.
暂无评论