In this paper, we study the convergence behavior of distributed iterative algorithms with quantized message passing. We first introduce a general iterative function evaluation algorithms for solving fixed point proble...
详细信息
ISBN:
(纸本)9781424497218
In this paper, we study the convergence behavior of distributed iterative algorithms with quantized message passing. We first introduce a general iterative function evaluation algorithms for solving fixed point problems distributively. We then analyze the convergence of the distributed algorithms, e.g. Jacobi scheme and Gauss-Seidel scheme, under the quantized message passing. Based on the closed-form convergence performance derived, we propose two quantizer designs, namely the time invariant convergence-optimal quantizer (TICOQ) and the time varying convergence-optimal quantizer (TVCOQ) to minimize the effect of the quantization error on the convergence. We also study the tradeoff between the convergence error and message passing overhead for both TICOQ and TVCOQ. As an example, we apply the TICOQ and TVCOQ designs to the iterative waterfilling algorithm of MIMO interference game.
Synchronous iterative algorithms are often less scalable than asynchronous iterative ones. Performing large scale experiments with different kind of network parameters is not easy because with supercomputers such para...
详细信息
ISBN:
(纸本)9781479961238
Synchronous iterative algorithms are often less scalable than asynchronous iterative ones. Performing large scale experiments with different kind of network parameters is not easy because with supercomputers such parameters are fixed. So, one solution consists in using simulations first in order to analyze what parameters could influence or not the behavior of an algorithm. In this paper, we show that it is interesting to use SimGrid to simulate the behavior of asynchronous iterative algorithms. For that, we compare the behavior of a synchronous GMRES algorithm with an asynchronous multisplitting one with simulations which let us easily choose some parameters. Both codes are real MPI codes and simulations allow us to see when the asynchronous multisplitting algorithm can be more efficient than the GMRES one to solve a 3D Poisson problem.
In this paper, we consider a generalized mixed equilibrium problem in real Hilbert space. Using the auxiliary principle, we define a class of resolvent mappings. Further, using fixed point and resolvent methods, we gi...
详细信息
In this paper, we consider a generalized mixed equilibrium problem in real Hilbert space. Using the auxiliary principle, we define a class of resolvent mappings. Further, using fixed point and resolvent methods, we give some iterative algorithms for solving generalized mixed equilibrium problem. Furthermore, we prove that the sequences generated by iterative algorithms converge weakly to the solution of generalized mixed equilibrium problem. These results require monotonicity ( θ -pseudo monotonicity) and continuity (Lipschitz continuity) for mappings.
Let C be a nonempty closed convex subset of a Banach space E with the dual E *, let T:C→E * be a Lipschitz continuous mapping and let S:C→C be a relatively nonexpansive mapping. In this paper, by employing the notio...
详细信息
The paper describes the FPGA technology together with its possibility to exploit spatial and temporal parallelism in order to implement hardware architectures for iterative algorithms. The development of hardware arch...
详细信息
ISBN:
(纸本)9781479904020;9781479904037
The paper describes the FPGA technology together with its possibility to exploit spatial and temporal parallelism in order to implement hardware architectures for iterative algorithms. The development of hardware architecture using FPGA technology represents a reliable solution in case of various applications where fast processing in case of iterative algorithms it's mandatory. Two applications are presented where the FPGA technology is used for processing. Thus, on one hand, automatic microarray grid alignment is performed using FPGA based hardware architecture, while on the other hand, an FPGA based LDPC decoder implementation is proposed in order to improve the decoder throughput compared to state of the art approaches.
In this paper, we address the design challenge of building multiresilient iterative high-performance computing (HPC) applications. Multiresilience in HPC applications is the ability to tolerate and maintain forward pr...
详细信息
ISBN:
(纸本)9783030105495;9783030105488
In this paper, we address the design challenge of building multiresilient iterative high-performance computing (HPC) applications. Multiresilience in HPC applications is the ability to tolerate and maintain forward progress in the presence of both soft errors and process failures. We address the challenge by proposing performance models which are useful to design performance efficient and resilient iterative applications. The models consider the interaction between soft error and process failure resilience solutions. We experimented with a linear solver application with two distinct kinds of soft error detectors: one detector has high overhead and high accuracy, whereas the second has low overhead and low accuracy. We show how both can be leveraged for verifying the integrity of checkpointed state used to recover from both soft errors and process failures. Our results show the performance efficiency and resiliency benefit of employing the low overhead detector with high frequency within the checkpoint interval, so that timely soft error recovery can take place, resulting in less re-computed work.
In statistical learning theory, generalization error is used to quantify the degree to which a supervised machine learning algorithm may overfit to training data. Recent work [Xu and Raginsky (2017)] has established a...
详细信息
ISBN:
(纸本)9781538647813
In statistical learning theory, generalization error is used to quantify the degree to which a supervised machine learning algorithm may overfit to training data. Recent work [Xu and Raginsky (2017)] has established a bound on the generalization error of empirical risk minimization based on the mutual information I(S;W) between the algorithm input S and the algorithm output W, when the loss function is sub-Gaussian. We leverage these results to derive generalization error bounds for a broad class of iterative algorithms that are characterized by bounded, noisy updates with Markovian structure. Our bounds are very general and are applicable to numerous settings of interest, including stochastic gradient Langevin dynamics (SGLD) and variants of the stochastic gradient Hamiltonian Monte Carlo (SGHMC) algorithm. Furthermore, our error bounds hold for any output function computed over the path of iterates, including the last iterate of the algorithm or the average of subsets of iterates, and also allow for non-uniform sampling of data in successive updates of the algorithm.
The importance of fault tolerance for parallel computing is ever increasing. The mean time between failures (MTBF) is predicted to decrease significantly for future highly parallel systems. At the same time, the curre...
详细信息
ISBN:
(纸本)9781479953134
The importance of fault tolerance for parallel computing is ever increasing. The mean time between failures (MTBF) is predicted to decrease significantly for future highly parallel systems. At the same time, the current trend to use commodity hardware to reduce the cost of clusters puts pressure on users to ensure fault tolerance of their applications. Cloud-based resources are one of the environments where the latter holds true. When it comes to embarrassingly parallel data-intensive algorithms, MapReduce has gone a long way in ensuring users can easily utilize these resources without the fear of losing work. However, this does not apply to iterative communication-intensive algorithms common in the scientific computing domain. In this work we propose a new programming model inspired by Bulk Synchronous Parallel (BSP), for creating a new fault tolerant distributed computing framework. We strive to retain the advantages that MapReduce provides, yet efficiently support a larger assortment of algorithms, such as the aforementioned iterative ones. The model adopts an approach similar to continuation passing for implementing parallel algorithms and facilitates fault tolerance inherent in the BSP program structure. Based on the model we created a distributed computing framework NEWT, which we describe and use to validate the approach.
Nonlinear regression models are more common as compared to linear ones for real life cases e. g. climatology, biology, earthquake engineering, economics etc. However, nonlinear regression models are much more complex ...
详细信息
Nonlinear regression models are more common as compared to linear ones for real life cases e. g. climatology, biology, earthquake engineering, economics etc. However, nonlinear regression models are much more complex to fit and to interpret. Classical parameter estimation methods such as least squares and maximum likelihood can also be adopted to fit the model in nonlinear regression as well, but explicit solutions can not be achieved unlike linear models. At this point, iterative algorithms are utilized to solve the problem numerically. Since there is no extensive study which compiles, classifies and compares the existing methods for nonlinear parameter estimation, the objective of this study is to fill this gap. In our study, we aim to compile the methods which are used for nonlinear parameter estimation purpose and compare them with respect to several criteria such as bias, execution time, number of iterations etc. The comparison will be conducted considering different scenarios which are small vs. large sample sizes, good vs. poor initial values, normal vs. non-normal error terms, simple vs complex models (with respect to number of parameters), and robustness. Both real and simulated data are used in the comparative study.
This paper addresses the problem of optical signal-to-noise ratio (OSNR) optimization problem in optical networks. Based on the extended OSNR Nash game formulation that includes power capacity constraints in [10], the...
详细信息
This paper addresses the problem of optical signal-to-noise ratio (OSNR) optimization problem in optical networks. Based on the extended OSNR Nash game formulation that includes power capacity constraints in [10], the Nash equilibrium (NE) solution is analytically intractable and highly nonlinear. We investigate the properties of the NE solution and based on these, we develop iterative algorithms to compute the NE solution: a parallel update algorithm (PUA) and a relaxed parallel update algorithm (r-PUA). We study their convergence with different conditions, both theoretically and numerically.
暂无评论