This paper considers the problem of decentralized, personal-ized federated learning. For centralized personalized federated learning, a penalty that measures the deviation from the local model and its average, is ofte...
详细信息
This paper considers the problem of decentralized, personal-ized federated learning. For centralized personalized federated learning, a penalty that measures the deviation from the local model and its average, is often added to the objective function. However, in a decentralized setting this penalty is expensive in terms of communication costs, so here, a different penalty - one that is built to respect the structure of the underlying computational network - is used instead. We present lower bounds on the communication and local computation costs for this problem formulation and we also present provably optimal methods for decentralized personalized federated learning. Numerical experiments are presented to demonstrate the practical performance of our methods. (c) 2022 The Authors. Published by Elsevier Ltd on behalf of Association of European Operational Research Societies (EURO). This is an open access article under the CC BY-NC-ND license (http://***/licenses/by-nc-nd/4.0/).
We consider composite minimax optimization problems where the goal is to find a saddle-point of a large sum of non -bilinear objective functions augmented by simple composite regularizers for the primal and dual varia...
详细信息
We consider composite minimax optimization problems where the goal is to find a saddle-point of a large sum of non -bilinear objective functions augmented by simple composite regularizers for the primal and dual variables. For such problems, under the average-smoothness assumption, we propose accelerated stochastic variance-reduced algorithms with optimal up to logarithmic factors complexity bounds. In particular, we consider strongly-convex-strongly-concave, convex-strongly-concave, and convex-concave objectives. To the best of our knowledge, these are the first nearly-optimal algorithms for this setting.(c) 2022 The Author(s). Published by Elsevier Ltd on behalf of Association of European Operational Research Societies (EURO). This is an open access article under the CC BY license (http://***/licenses/by/4.0/).
The low-rank plus sparse (L+S) decomposition model enables the reconstruction of undersampled dynamic parallel magnetic resonance imaging data. Solving for the low rank and the sparse components involves nonsmooth com...
详细信息
The low-rank plus sparse (L+S) decomposition model enables the reconstruction of undersampled dynamic parallel magnetic resonance imaging data. Solving for the low rank and the sparse components involves nonsmooth composite convex optimization, and algorithms for this problem can he categorized into proximal gradient methods and variable splitting methods. This paper investigates new efficient algorithms for both schemes. While current proximal gradient techniques for the L+S model involve the classical iterative soft thresholding algorithm (ISTA), this paper considers two accelerated alternatives, one based on the fast iterative shrinkage-thresholding algorithm (FISTA) and the other with the recent proximal optimized gradient method (POGM). In the augmented Lagrangian (AL) framework, we propose an efficient variable splitting scheme based on the form of the data acquisition operator, leading to simpler computation than the conjugate gradient approach required by existing AL methods. Numerical results suggest faster convergence of the efficient implementations for both frameworks, with POGM providing the fastest convergence overall and the practical benefit of being free of algorithm tuning parameters.
The microscopic spatial kinetic Monte Carlo (KMC) method has been employed extensively in materials modeling. In this review paper, we focus on different traditional and multiscale KMC algorithms, challenges associate...
详细信息
The microscopic spatial kinetic Monte Carlo (KMC) method has been employed extensively in materials modeling. In this review paper, we focus on different traditional and multiscale KMC algorithms, challenges associated with their implementation, and methods developed to overcome these challenges. In the first part of the paper, we compare the implementation and computational cost of the null-event and rejection-free microscopic KMC algorithms. A firmer and more general foundation of the null-event KMC algorithm is presented. Statistical equivalence between the null-event and rejection-free KMC algorithms is also demonstrated. Implementation and efficiency of various search and update algorithms, which are at the heart of all spatial KMC simulations, are outlined and compared via numerical examples. In the second half of the paper, we review various spatial and temporal multiscale KMC methods, namely, the coarse-grained Monte Carlo (CGMC), the stochastic singular perturbation approximation, and the tau-leap methods, introduced recently to overcome the disparity of length and time scales and the one-at-a time execution of events. The concepts of the CGMC and the tau-leap methods, stochastic closures, multigrid methods, error associated with coarse-graining, a posteriori error estimates for generating spatially adaptive coarse-grained lattices, and computational speed-up upon coarse-graining are illustrated through simple examples from crystal growth, defect dynamics, adsorption-desorption, surface diffusion, and phase transitions.
A technique to accelerate convergence of stochastic approximation algorithms is studied. It is based on Kesten's idea of equalization of the gain coefficient for the Robbins-Monro algorithm. Convergence with proba...
详细信息
A technique to accelerate convergence of stochastic approximation algorithms is studied. It is based on Kesten's idea of equalization of the gain coefficient for the Robbins-Monro algorithm. Convergence with probability is proved for the multidimensional analog of the Kesten accelerated stochastic approximation algorithm. Asymptotic normality of the delivered estimates is also shown. Results of numerical simulations are presented that demonstrate the efficiency of the acceleration procedure.
This paper presents a method of classification taking account of a contiguity constraint. This procedure is applicable to all data sets represented in two distinct spaces, one of which is defined by the introduction o...
详细信息
This paper presents a method of classification taking account of a contiguity constraint. This procedure is applicable to all data sets represented in two distinct spaces, one of which is defined by the introduction of a relation of contiguity. The basic idea is to favour the pairs of clusters which are close in this space (i.e. contiguous). In practice, the hierarchy depends on the choice of a contiguity threshold equal to the maximal distance beyond which two points are not contiguous. Following the description of the algorithm, one will find results dealing with the influence of the constraint and with the estimation of the contiguity threshold. To illustrate this paper, classifications of ground points distributed on a plane area and characterized by geochemical descriptors are given.
暂无评论