In this work, we consider a sequence of J matrix multiplication jobs which needs to be distributed by a master across multiple worker nodes. For i is an element of{1, 2,..., J}, job-i begins in round-i and has to be c...
详细信息
In this work, we consider a sequence of J matrix multiplication jobs which needs to be distributed by a master across multiple worker nodes. For i is an element of{1, 2,..., J}, job-i begins in round-i and has to be completed by round-(i + T). In order to provide resiliency against slow workers (stragglers), previous works focus on coding across workers, which is the special case of T = 0. We propose here two schemes with T > 0, which allow for coding across workers as well as the dimension of time. Our first scheme is a modification of the polynomial coding scheme introduced by Yu et al. and places no assumptions on the straggler model. Exploitation of the temporal dimension helps the scheme handle a larger set of straggler patterns than the polynomial coding scheme, for a given computational load per worker per round. The second scheme assumes a particular straggler model to further improve performance (in terms of encoding/decoding complexity). We develop theoretical results establishing (i) optimality of our proposed schemes for certain classes of straggler patterns and (ii) improved performance for the case of i.i.d. stragglers. These are further validated by experiments, where we implement our schemes to train neural networks.
Coded computing is an effective technique to mitigate "stragglers" in large-scale and distributed matrix multiplication. In particular, univariate polynomial codes have been shown to be effective in straggle...
详细信息
Coded computing is an effective technique to mitigate "stragglers" in large-scale and distributed matrix multiplication. In particular, univariate polynomial codes have been shown to be effective in straggler mitigation by making the computation time depend only on the fastest workers. However, these schemes completely ignore the work done by the straggling workers resulting in a waste of computational resources. To reduce the amount of work left unfinished at workers, one can further decompose the matrix multiplication task into smaller sub-tasks, and assign multiple sub-tasks to each worker, possibly heterogeneously, to better fit their particular storage and computation capacities. In this work, we propose a novel family of bivariate polynomial codes to efficiently exploit the work carried out by straggling workers. We show that bivariate polynomial codes bring significant advantages in terms of upload communication costs and storage efficiency, measured in terms of the number of sub-tasks that can be computed per worker. We propose two bivariate polynomial coding schemes. The first one exploits the fact that bivariate interpolation is always possible on a rectangular grid of evaluation points. We obtain such points at the cost of adding some redundant computations. For the second scheme, we relax the decoding constraints and require decodability for almost all choices of the evaluation points. We present interpolation sets satisfying such decodability conditions for certain storage configurations of workers. Our numerical results show that bivariate polynomial coding considerably reduces the average computation time of distributed matrix multiplication. We believe this work opens up a new class of previously unexplored coding schemes for efficient coded distributed computation.
Particle filters have been widely used for the solution of optimal estimation problems in nonlinear non-Gaussian environments. One of their drawbacks is that these methods are computationally expensive. In the past fe...
详细信息
Particle filters have been widely used for the solution of optimal estimation problems in nonlinear non-Gaussian environments. One of their drawbacks is that these methods are computationally expensive. In the past few years, new developments have been made in trying to distribute the particle filter algorithm among different computing agents in order to make the underlying computations tractable. This period also witnessed the rise of general purpose GPU devices, which are making massive code parallelization possible. These developments have the potential to make the particle filter a viable alternative for real-time implementations in the near future, even when the number of required particles is high. In this paper we review the state-of-the-art in distributed particle filtering and propose a method that is applicable to distributed computing architectures.
A distributed virtual reality system is presented based on a hybrid rendering scheme. This system first predicts the camera position of the next key frame in real-time according to the tendency of the current camera m...
详细信息
A distributed virtual reality system is presented based on a hybrid rendering scheme. This system first predicts the camera position of the next key frame in real-time according to the tendency of the current camera movement. The predicted key frame is then generated using a geometry-based algorithm. Finally, a real-time backward image warping algorithm is employed to create a smooth transition between the two adjacent key frames. Since the rendering procedure can be divided into two independent steps, this system can be efficiently implemented in distributed mode.
In recent years sensor networks have interested fields such as environment monitoring, surveillance and other distributed applications for data elaboration. This interest has been based on the decentralized approach i...
详细信息
In recent years sensor networks have interested fields such as environment monitoring, surveillance and other distributed applications for data elaboration. This interest has been based on the decentralized approach in treating the information. However it is still a challenge to manipulate such streams of data when the dimension of the net becomes large despite computational capabilities and consumption constraints. In most of applications, location awareness is fundamental to accomplish common tasks. In this paper a probabilistic approach to solve localization problem in wireless sensor networks is presented. The algorithm, based on the Kalman Filter, estimates the sensors' location by an adaptive behavior. The technique proposed allows a reduction of the computation burden respect to the traditional Kalman Filter showing, as explained in simulations and real world experiments, good performances.
In this paper, we compare six known linear distributed average consensus algorithms on a sensor network in terms of convergence time (and therefore, in terms of the number of transmissions required). The selected netw...
详细信息
In this paper, we compare six known linear distributed average consensus algorithms on a sensor network in terms of convergence time (and therefore, in terms of the number of transmissions required). The selected network topologies for the analysis (comparison) are the cycle and the path. Specifically, in the present paper, we compute closed-form expressions for the convergence time of four known deterministic algorithms and closed-form bounds for the convergence time of two known randomized algorithms on cycles and paths. Moreover, we also compute a closed-form expression for the convergence time of the fastest deterministic algorithm considered on grids.
Sequential LQG algorithms are formulated for stochastic linear discrete-time systems having a Lower Block Triangular (LBT) decomposition. Three distinct information structures are discussed from the point of view of c...
详细信息
Sequential LQG algorithms are formulated for stochastic linear discrete-time systems having a Lower Block Triangular (LBT) decomposition. Three distinct information structures are discussed from the point of view of control and estimation design and implementation
作者:
T. BaşarDecision and Control Laboratory
Coordinated Science Laboratory and the Department of Electrical and Computer Engineering University of Illionis 1101W Springfield Avenue Urbana IL 61801 USA
The main objective of this paper is to review some recent results and also to present some new algorithms for the on-line, distributed computation of equilibria in noncooperative deterministic and stochastic games. Th...
详细信息
The main objective of this paper is to review some recent results and also to present some new algorithms for the on-line, distributed computation of equilibria in noncooperative deterministic and stochastic games. The algorithms to be studied are of the relaxation-type , which require memory on either the past policy choices or the past actions of the players, depending on whether the iteration (and learning) is in the policy or the action space. Our earlier work on this topic has shown that incorporating memory into Jacobi and Gauss-Seidel type iterations in two-person Nash games could lead to considerable improvements in both the region of convergence and the speed of convergence towards noncooperative equilibrium. under both asynchronous and totally asynchronous modes of update. The present paper shows that these appealing features carry over to partially asynchronous modes of update, when there is a bound on the memory available to one or more players. The paper also discusses robustness of the partially asynchronous relaxation-type algorithms to inaccuracies in each player's knowledge regarding the others' response functions.
The work gives a review on the distributed Nash equilibrium seeking of noncooperative games in multi-agent networks,which emerges as one of the frontier research topics in the area of systems and control ***,we give t...
详细信息
The work gives a review on the distributed Nash equilibrium seeking of noncooperative games in multi-agent networks,which emerges as one of the frontier research topics in the area of systems and control ***,we give the basic formulation and analysis of noncooperative games with continuous action spaces,and provide the motivation and basic setting for distributed Nash equilibrium *** we introduce both the gradient-based algorithms and best-response based algorithms for various type of games,including zero-sum games,aggregative games,potential games,monotone games,and multi-cluster *** addition,we provide some applications of noncooperative games.
Design of nonlinear dynamic systems is a challenging task. Computers may help to analyze and optimize virtual prototypes in an earl design phase, but there is a lack of design tools. In this paper, a design concept on...
详细信息
Design of nonlinear dynamic systems is a challenging task. Computers may help to analyze and optimize virtual prototypes in an earl design phase, but there is a lack of design tools. In this paper, a design concept on the basis of a multibody system approach is described. Programs for modeling, simulation, sensitivity analysis, optimization, and animation are operating under a common administration which ensures model consistency. The designer is supported by a graphical user interface, distributed computation improves efficiency.
暂无评论