检索结果-内蒙古大学图书馆

Quantized Distributed Nonconvex Optimization with Linear Convergence

学校读者我要写书评

暂无评论

Quantized Distributed Nonconvex Optimization with Linear Con...

IEEE Conference on Decision and control

作者： Lei Xu Xinlei Yi Jiayue Sun Yang Shi Karl Henrik Johansson Tao Yang State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang China Division of Decision and Control Systems School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Digital Futures Stockholm Sweden Department of Mechanical Engineering University of Victoria Victoria Canada

ISBN: (数字)9781665467612

ISBN: (纸本)9781665467629

This paper considers distributed nonconvex op-timization for minimizing the average of local cost functions, by using local information exchange over undirected communication networks. Since the communication channels often have limited bandwidth or capacity, we first introduce a quantization rule and an encoder/decoder scheme to reduce the transmission bits. By integrating them with a distributed algorithm, we then propose a distributed quantized nonconvex optimization algorithm. Assuming the global cost function satisfies the Polyak– Łojasiewicz condition, which does not require the global cost function to be convex and the global minimizer is not necessarily unique, we show that the proposed algorithm linearly converges to a global optimal point. Moreover, a low data rate is shown to be sufficient to ensure linear convergence when the algorithm parameters are properly chosen. The theoretical results are illustrated by numerical simulation examples.

关键词： Quantization (signal) Communication channels Bandwidth Cost function Numerical simulation Communication networks Distributed algorithms

Distributed Least Squares Solver for Network Linear Equations

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Yang, Tao George, Jemin Qin, Jiahu Yi, Xinlei Wu, Junfeng State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang110819 China U.S. Army Research Laboratory AdelphiMD20783 United States Department of Automation University of Science and Technology of China Hefei230027 China Division of Decision and Control Systems School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Stockholm100 44 Sweden College of Control Science and Engineering Zhejiang University Hangzhou310027 China

In this paper, we study the problem of finding the least square solutions of over-determined linear algebraic equations over networks in a distributed manner. Each node has access to one of the linear equations and holds a dynamic state. We first propose a distributed least square solver over connected undirected interaction graphs and establish a necessary and sufficient on the step-size under which the algorithm exponentially converges to the least square solution. Next, we develop a distributed least square solver over strongly connected directed graphs and show that the proposed algorithm exponentially converges to the least square solution provided the step-size is sufficiently small. Moreover, we develop a finite-time least square solver by equipping the proposed algorithms with a finite-time decentralized computation mechanism. The theoretical findings are validated and illustrated by numerical simulation examples. Copyright © 2018, The Authors. All rights reserved.

关键词： Linear equations

Zeroth-order algorithms for stochastic distributed nonconvex optimization

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Yi, Xinlei Zhang, Shengjun Yang, Tao Johansson, Karl H. Division of Decision and Control Systems School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Stockholm10044 Sweden Department of Electrical Engineering University of North Texas DentonTX76203 United States State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang110819 China

In this paper, we consider a stochastic distributed nonconvex optimization problem with the cost function being distributed over n agents having access only to zeroth-order (ZO) information of the cost. This problem has various machine learning applications. As a solution, we propose two distributed ZO algorithms, in which at each iteration each agent samples the local stochastic ZO oracle at two points with a time-varying smoothing parameter. We show that the proposed algorithms achieve the linear speedup convergence rate O(√p=(nT)) for smooth cost functions under the state-dependent variance assumptions which are more general than the commonly used bounded variance and Lipschitz assumptions, and O(p=(nT)) convergence rate when the global cost function additionally satisfies the Polyak-Lojasiewicz (P-L) condition in addition, where p and T are the dimension of the decision variable and the total number of iterations, respectively. To the best of our knowledge, this is the first linear speedup result for distributed ZO algorithms, which enables systematic processing performance improvements by adding more agents. We also show that the proposed algorithms converge linearly under the relative bounded second moment assumptions and the P-L condition. We demonstrate through numerical experiments the efficiency of our algorithms on generating adversarial examples from deep neural networks in comparison with baseline and recently proposed centralized and distributed ZO algorithms. Copyright © 2021, The Authors. All rights reserved.

关键词： Cost functions

Recursive Weighted Null-Space Fitting Method for Identification of Multivariate Systems ⁎

学校读者我要写书评

暂无评论

IFAC-PapersOnLine 2021年第7期54卷 345-350页

作者： Mengyuan Fang Miguel Galrinho Håkan Hjalmarsson Zhejiang Engineering Laboratory of Industrial Internet Communication Technology Zhejiang Sci-Tech University Hangzhou China. State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang China OHB Sweden AB Kista Sweden Centre for Advanced Bio Production and Division of Decision and Control Systems School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Sweden

Recursive identification of structured multivariate models is known to be difficult due to the general non-convexity of the likelihood function. In this work, we propose a recursive multivariate weighted null-space fitting method for identification of structured multivariate models. The proposed method first uses recursive least squares to estimate a high order non-parametric model, then a parametric model is obtained through weighted least squares from the non-parametric model. In this way, the method avoids directly optimizing a non-convex likelihood function and has guaranteed global convergency. Moreover, the proposed method is flexible in model structures and has the same finite sample performance as its off-line counterpart. We use simulation examples to illustrate the performance.

关键词： Recursive system identification multivariate systems weighted null-space fitting

A Primal-Dual SGD Algorithm for Distributed Nonconvex Optimization

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Yi, Xinlei Zhang, Shengjun Yang, Tao Chai, Tianyou Johansson, Karl H. The Division of Decision and Control Systems School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Stockholm100 44 Sweden The Department of Electrical Engineering University of North Texas DentonTX76203 United States The State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang110819 China

The distributed nonconvex optimization problem of minimizing a global cost function formed by a sum of n local cost functions by using local information exchange is considered. This problem is an important component of many machine learning techniques with data parallelism, such as deep learning and federated learning. We propose a distributed primal–dual stochastic gradient descent (SGD) algorithm, suitable for arbitrarily connected communication networks and any smooth (possibly nonconvex) cost functions. We show that the proposed algorithm achieves the linear speedup convergence rate O(1/√nT) for general nonconvex cost functions and the linear speedup convergence rate O(1/(nT)) when the global cost function satisfies the Polyak–Lojasiewicz (P–L) condition, where T is the total number of iterations. We also show that the output of the proposed algorithm with constant parameters linearly converges to a neighborhood of a global optimum. We demonstrate through numerical experiments the efficiency of our algorithm in comparison with the baseline centralized SGD and recently proposed distributed SGD algorithms. Copyright © 2020, The Authors. All rights reserved.

关键词： Cost functions

Exponential convergence for distributed smooth optimization under the restricted secant inequality condition

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Yi, Xinlei Zhang, Shengjun Yang, Tao Johansson, Karl H. Chai, Tianyou Division of Decision and Control Systems School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Stockholm100 44 Sweden Department of Electrical Engineering University of North Texas DentonTX76203 United States State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang110819 China

This paper considers the distributed smooth optimization problem in which the objective is to minimize a global cost function formed by a sum of local smooth cost functions, by using local information exchange. The standard assumption for proving exponential/linear convergence of first-order methods is the strong convexity of the cost functions, which does not hold for many practical applications. In this paper, we first show that the continuous-time distributed primal-dual gradient algorithm converges to one global minimizer exponentially under the assumption that the global cost function satisfies the restricted secant inequality condition. This condition is weaker than the strong convexity condition since it does not require convexity and the global minimizers are not necessary to be unique. We then show that the discrete-time distributed primal-dual algorithm constructed by using the Euler's approximation method converges to one global minimizer linearly under the same condition. The theoretical results are illustrated by numerical simulations. Copyright © 2019, The Authors. All rights reserved.

关键词： Continuous time systems

An atraumatic mock loop for realistic hemocompatibility assessment of blood pumps

学校读者我要写书评

暂无评论

TechRxiv

TechRxiv 2023年

作者： Bender, Moritz Escher, Andreas Röhrich, Michael Fischer, Michael Bernhard Hametner, Christoph Laufer, Günther Kertzscher, Ulrich Zimpfer, Daniel Jakubek, Stefan Granegger, Marcus The Department of Cardiac Surgery Medical University of Vienna Austria The Division of Control and Process Automation Institute of Mechanics and Mechatronics Vienna University of Technology Vienna1040 Austria Biofluid Mechanics Laboratory Berlin Germany The Division of Special Anesthesia and Pain Medicine Department of Anesthesia Intensive Care Medicine and Pain Medicine Medical University of Vienna Austria The Department for Blood Group Serology and Transfusion Medicine Medical University of Vienna Austria

Conventional mock circulatory loops (MCLs) cannot replicate realistic hemodynamic conditions without inducing blood trauma. This constrains in-vitro hemocompatibility examinations of blood pumps to static operating conditions that do not reflect clinical scenarios. This study aimed at developing an atraumatic MCL based on a hardware-in-the-loop concept (H-MCL) that enables hemocompatibility assessment under realistic hemodynamic conditions. The atraumatic H-MCL was designed for 450±50 ml of blood with the sole blood-contacting components being the polycarbonate reservoirs, the silicone/polyvinyl-chloride tubing and the blood pump under investigation. To account for inherent coupling effects a decoupling pressure control was derived by feedback linearization, whereas the level control was addressed by an optimization task to overcome periodic loss of controllability. The HeartMate 3 was showcased to evaluate the H-MCL’s accuracy at typical hemodynamic conditions and to demonstrate the H-MCL’s atraumatic properties. Pilot hemolysis experiments were conducted with bovine (n=2) and human (n=2) blood, and evaluated in terms of the normalized index of hemolysis (NIH). Typical hemodynamic scenarios of patients with full and partial support were replicated with marginal coupling effects and root mean square error (RMSE) of 1.74±1.37 mmHg and 0.94±0.83 mmHg, respectively, while the fluid level did not exceed a range of ±4% of its target value. The initial examinations of hemolysis demonstrated the atraumatic characteristics of the novel H-MCL, as evidenced by levels of NIH (bovine: 5.1-7.2mg/100L;human: 1.6-1.8mg/100L) that are consistent with those reported in existing literature. Collectively, these findings indicated the H-MCL’s potential for in-vitro hemocompatibiltiy assessment of blood pumps within realistic hemodynamic conditions. © 2023, CC BY.

关键词： Hemodynamics

Communication Compression for Distributed Nonconvex Optimization

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yi, Xinlei Zhang, Shengjun Yang, Tao Chai, Tianyou Johansson, Karl H. Division of Decision and Control Systems School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Digital Futures Stockholm100 44 Sweden Department of Electrical Engineering University of North Texas DentonTX76203 United States State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang110819 China

This paper considers distributed nonconvex optimization with the cost functions being distributed over agents. Noting that information compression is a key tool to reduce the heavy communication load for distributed algorithms as agents iteratively communicate with neighbors, we propose three distributed primal–dual algorithms with compressed communication. The first two algorithms are applicable to a general class of compressors with bounded relative compression error and the third algorithm is suitable for two general classes of compressors with bounded absolute compression error. We show that the proposed distributed algorithms with compressed communication have comparable convergence properties as state-of-the-art algorithms with exact communication. Specifically, we show that they can find first-order stationary points with sublinear convergence rate O(1/T) when each local cost function is smooth, where T is the total number of iterations, and find global optima with linear convergence rate under an additional condition that the global cost function satisfies the Polyak–Lojasiewicz condition. Numerical simulations are provided to illustrate the effectiveness of the theoretical results. Copyright © 2022, The Authors. All rights reserved.

关键词： Iterative methods

Linear convergence of first- And zeroth-order primal-dual algorithms for distributed nonconvex optimization

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Yi, Xinlei Zhang, Shengjun Yang, Tao Chai, Tianyou Johansson, Karl H. Division of Decision and Control Systems School of Electrical Engineering and Computer Science Kth Royal Institute of Technology Digital Futures Stockholm100 44 Sweden Department of Electrical Engineering University of North Texas DentonTX76203 United States State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang110819 China

This paper considers the distributed nonconvex optimization problem of minimizing a global cost function formed by a sum of local cost functions by using local information *** first propose a distributed first-order primal-dual algorithm. We show that it converges sublinearly to a stationary point if each local cost function is smooth and linearly to a global optimum under an additional condition that the global cost function satisfies the Polyak-Lojasiewicz condition. This condition is weaker than strong convexity, which is a standard condition for proving linear convergence of distributed optimization algorithms, and the global minimizer is not necessarily unique. Motivated by the situations where the gradients are unavailable, we then propose a distributed zeroth-order algorithm, derived from the proposed distributed first-order algorithm by using a deterministic gradient estimator, and show that it has the same convergence properties as the proposed first-order algorithm under the same conditions. The theoretical results are illustrated by numerical simulations. Copyright © 2019, The Authors. All rights reserved.

关键词： Cost functions