检索结果-内蒙古大学图书馆

A Primal-Dual SGD Algorithm for Distributed Nonconvex Optimization

IEEE/CAA Journal of Automatica Sinica 2022年第5期9卷 812-833页

作者： Xinlei Yi Shengjun Zhang Tao Yang Tianyou Chai Karl Henrik Johansson Division of Decision and Control Systems School of Electrical Engineering and Computer ScienceKTH Royal Institute of Technologyand also affiliated with the Digital FuturesStockholm 10044Sweden Department of Electrical Engineering University of North TexasDentonTX 76203 USA IEEE State Key Laboratory of Synthetical Automation for Process Industries Northeastern UniversityShenyang 110819China

The distributed nonconvex optimization problem of minimizing a global cost function formed by a sum of n local cost functions by using local information exchange is *** problem is an important component of many machine learning techniques with data parallelism,such as deep learning and federated *** propose a distributed primal-dual stochastic gradient descent(SGD)algorithm,suitable for arbitrarily connected communication networks and any smooth(possibly nonconvex)cost *** show that the proposed algorithm achieves the linear speedup convergence rate O(1/(√nT))for general nonconvex cost functions and the linear speedup convergence rate O(1/(nT)) when the global cost function satisfies the Polyak-Lojasiewicz(P-L)condition,where T is the total number of *** also show that the output of the proposed algorithm with constant parameters linearly converges to a neighborhood of a global *** demonstrate through numerical experiments the efficiency of our algorithm in comparison with the baseline centralized SGD and recently proposed distributed SGD algorithms.

关键词： Distributed nonconvex optimization linear speedup Polyak-Lojasiewicz(P-L)condition primal-dual algorithm stochastic gradient descent

来源：评论

学校读者我要写书评

暂无评论

EOG acquisition system based on ATmega AVR microcontroller

引用

Journal of Ambient Intelligence and Humanized Computing 2023年第12期14卷 16589-16605页

作者： Abdel-Samei, Abdel-Gawad A. Shaaban, Abdel-Salam Brisha, Ayman M. El-Samie, Fathi E. Abd Ali, Ahmed S. Department of Electronics Technology Faculty of Technology and Education Beni-Suef University Beni-Suef Egypt Department of Process of Control Technology Faculty of Technology and Education Beni-Suef University Beni-Suef Egypt Department of Mechanical Engineering Mechatronics Division Faculty of Engineering Assiut University Assiut Egypt Department of Electronics and Electrical Communications Engineering Faculty of Electronic Engineering Menoufia University Menouf Egypt Department of Information Technology College of Computer and Information Sciences Princess Nourah Bint Abdulrahman University P.O. Box 84428 Riyadh11671 Saudi Arabia

This paper presents a hardware and software of an electrooculogram (EOG) acquisition system based on ATmega AVR microcontroller for the acquisition of vertical and horizontal eye movements. The presented system is used to acquire a dataset of eye movements for volunteers. This system gives two channels representing vertical and horizontal EOG signals. The frequency range of the EOG signal is known to be 0.1 to 10 Hz, and hence this frequency range is isolated with a High-Pass Filter (HPF) with a cutoff frequency of 0.1 Hz followed by a Low-Pass Filter (LPF) with a cutoff frequency of 10 Hz. The EOG acquisition system is interfaced with an ATmega AVR microcontroller to acquire a dataset that can be used for controlling hardware such as Light Emitting Diodes (LEDs), wheelchair, and robot arm. The presented system is composed of EOG signal acquisition, Ag/AgCl electrodes, analog-to-digital converter through Arduino Mega 2560 board microcontroller unit, trainer board, laptop, keypad, and Liquid Crystal Display (LCD). The eye movement is detected by measuring the potential difference between cornea and retina using five Ag–Agcl disposable electrodes. Different volunteers of different ages at different times have been treated with the presented system to obtain data. Classified vertical and horizontal EOG signals and the basic eye movements e.g., open eye, left, right, up, and down can be used to control robots and wheelchairs for rehabilitation purposes. © 2023, The Author(s).

关键词： Electrodes

来源：评论

学校读者我要写书评

暂无评论

Quantized Distributed Nonconvex Optimization Algorithms with Linear Convergence under the Polyak–Lojasiewicz Condition

引用

IEEE Transactions on Automatic control 2025年

作者： Xu, Lei Yi, Xinlei Sun, Jiayue Shi, Yang Johansson, Karl H. Yang, Tao Northeastern University State Key Laboratory of Synthetical Automation for Process Industries Shenyang110819 China University of Victoria Department of Mechanical Engineering VictoriaBCV8W 2Y2 Canada Tongji University Shanghai Institute of Intelligent Science and Technology Shanghai201804 China KTH Royal Institute of Technology Division of Decision and Control Systems School of Electrical Engineering and Computer Science Stockholm100 44 Sweden

This paper considers distributed optimization for minimizing the average of local nonconvex cost functions, by using local information exchange over undirected communication networks. To reduce the required communication capacity, we introduce an encoder–decoder scheme. By integrating it with distributed gradient tracking and proportional integral algorithms, respectively, we then propose two quantized distributed nonconvex optimization algorithms. Assuming the global cost function satisfies the Polyak–Lojasiewicz condition, which does not require the global cost function to be convex and the global minimizer is not necessarily unique, we show that our proposed algorithms linearly converge to a global optimal point. Moreover, we show that a low data rate is sufficient to guarantee linear convergence when the algorithm parameters are properly chosen. The theoretical results are illustrated by numerical examples. © 1963-2012 IEEE.

关键词： Cost functions

来源：评论

学校读者我要写书评

暂无评论

Reduced Network Cumulative Constraint Violation for Distributed Bandit Convex Optimization under Slater’s Condition

arXiv

引用

arXiv 2024年

作者： Zhang, Kunpeng Yi, Xinlei Ding, Jinliang Cao, Ming Johansson, Karl H. Yang, Tao State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang110819 China Department of Control Science and Engineering College of Electronics and Information Engineering Tongji University Shanghai201800 China Engineering and Technology Institute Groningen Faculty of Science and Engineering University of Groningen GroningenAG 9747 Netherlands Division of Decision and Control Systems School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Sweden Digital Futures Stockholm10044 Sweden

This paper studies the distributed bandit convex optimization problem with time-varying inequality constraints, where the goal is to minimize network regret and cumulative constraint violation. To calculate network cumulative constraint violation, existing distributed bandit online algorithms solving this problem directly use the clipped constraint function to replace its original constraint function. However, the use of the clipping operation renders Slater’s condition (i.e, there exists a point that strictly satisfies the inequality constraints at all iterations) ineffective to achieve reduced network cumulative constraint violation. To tackle this challenge, we propose a new distributed bandit online primal–dual algorithm. If local loss functions are convex, we show that the proposed algorithm establishes an O(T max{c,1−c})network regret bound and an O(T 1−c/2) network cumulative constraint violation bound, where T is the total number of iterations and c ∈ (0, 1) is a user-defined trade-off parameter. When Slater’s condition holds, the network cumulative constraint violation bound is reduced to O(T 1−c). In addition, if local loss functions are strongly convex, for the case where strongly convex parameters are unknown, the network regret bound is reduced to O(T1−c), and the network cumulative constraint violation bound is O(T1−c/2) and O(T1−c) without and with Slater’s condition, respectively. For the case where strongly convex parameters are known, the network regret bound is further reduced to O)(log(T)), and the network cumulative constraint violation bound is reduced to O(log(T)T) and O(log(T)) without and with Slater’s condition, respectively. To the best of our knowledge, this paper is among the first to establish reduced (network) cumulative constraint violation bounds for (distributed) bandit convex optimization with time-varying constraints under Slater’s condition. Finally, a numerical example is provided to verify the theoretical results. Copyright © 2

关键词： Convex optimization

来源：评论

学校读者我要写书评

暂无评论

Distributed Event-Triggered Bandit Convex Optimization with Time-Varying Constraints

arXiv

引用

arXiv 2024年

作者： Zhang, Kunpeng Yi, Xinlei Wen, Guanghui Cao, Ming Johansson, Karl H. Chai, Tianyou Yang, Tao The State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang110819 China The Laboratory for Information and Decision Systems Massachusetts Institute of Technology CambridgeMA02139 United States The Department of Mathematics Southeast University Nanjing210096 China The Engineering and Technology Institute Groningen Faculty of Science and Engineering University of Groningen GroningenAG 9747 Netherlands Division of Decision and Control Systems School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Sweden Digital Futures Stockholm10044 Sweden

This paper considers the distributed bandit convex optimization problem with time-varying inequality constraints over a network of agents, where the goal is to minimize network regret and cumulative constraint violation. Existing distributed online algorithms require that each agent broadcasts its decision to its neighbors at each iteration. To better utilize the limited communication resources, we propose a distributed event-triggered online primal-dual algorithm with two-point bandit feedback. Under several classes of appropriately chosen decreasing parameter sequences and non-increasing event-triggered threshold sequences, we establish dynamic network regret and network cumulative constraint violation bounds. These bounds are comparable to the results achieved by distributed event-triggered online algorithms with full-information feedback. Finally, a numerical example is provided to verify the theoretical results. Copyright © 2024, The Authors. All rights reserved.

关键词： Convex optimization

来源：评论

学校读者我要写书评

暂无评论

Distributed Event-Triggered Bandit Convex Optimization With Time-Varying Constraints

引用

IEEE Transactions on control of Network Systems 2025年

作者： Zhang, Kunpeng Yi, Xinlei Wen, Guanghui Cao, Ming Johansson, Karl H. Chai, Tianyou Yang, Tao Northeastern University State Key Laboratory of Synthetical Automation for Process Industries Shenyang110819 China Ministry of Education Shanghai Institute of Intelligent Science and Technology National Key Laboratory of Autonomous Intelligent Unmanned Systems Frontiers Science Center for Intelligent Autonomous Systems Shanghai201210 China Southeast University School of Automation Nanjing210096 China University of Groningen Engineering and Technology Institute Groningen Faculty of Science and Engineering GroningenAG 9747 Netherlands KTH Royal Institute of Technology Division of Decision and Control Systems School of Electrical Engineering and Computer Science Stockholm10044 Sweden KTH Royal Institute of Technology Digital Futures Stockholm10044 Sweden

This paper considers the distributed bandit convex optimization problem with time-varying inequality constraints over a network of agents, where the goal is to minimize network regret and cumulative constraint violation. Existing distributed online algorithms solving this problem require that each agent broadcasts its decision to its neighbors at each iteration. However, communication resources are often limited. To better utilize communication resources, we propose a distributed event-triggered online primal-dual algorithm with two-point bandit feedback. Under several classes of appropriately chosen decreasing parameter sequences and non-increasing event-triggered threshold sequences, we establish dynamic network regret and network cumulative constraint violation bounds. These bounds are comparable to the results achieved by distributed event-triggered online algorithms with full-information feedback. Finally, a numerical example is provided to verify the theoretical results. © 2014 IEEE.

关键词： Optimization algorithms

来源：评论

学校读者我要写书评

暂无评论

Distributed Online Convex Optimization with Time-Varying Constraints: Tighter Cumulative Constraint Violation Bounds under Slater's Condition

引用

IEEE Transactions on Automatic control 2025年

作者： Yi, Xinlei Li, Xiuxian Yang, Tao Xie, Lihua Hong, Yiguang Chai, Tianyou Johansson, Karl H. National Key Laboratory of Autonomous Intelligent Unmanned Systems Shanghai Institute of Intelligent Science and Technology China Ministry of Education Frontiers Science Center for Intelligent Autonomous Systems China Massachusetts Institute of Technology Lab for Information & Decision Systems CambridgeMA02139 United States Northeastern University State Key Laboratory of Synthetical Automation for Process Industries Shenyang110819 China Nanyang Technological University School of Electrical and Electronic Engineering 50 Nanyang Avenue 639798 Singapore Kth Royal Institute of Technology Division of Decision and Control Systems School of Electrical Engineering and Computer Science Sweden Digital Futures Stockholm10044 Sweden

This paper considers distributed online convex optimization with time-varying constraints. In this setting, a network of agents makes decisions at each round, and then only a portion of the loss function and a coordinate block of the constraint function are privately revealed to each agent. The loss and constraint functions are convex and can vary arbitrarily across rounds. The agents collaborate to minimize static network regret and network cumulative constraint violation. A novel distributed online algorithm with a vanishing stepsize is proposed and it achieves an O(Tmax {c,1-c}) static network regret bound and an O(T1-c/2) network cumulative constraint violation bound, where T is the number of rounds and c∈ (0,1) is a user-defined trade-off parameter. When Slater's condition holds (i.e, there is a point that strictly satisfies the inequality constraints), the network cumulative constraint violation bound is reduced to O(T1-c). Moreover, if the loss functions are strongly convex, then static network regret bound is reduced to O(log (T)), and the network cumulative constraint violation bound is reduced to O(√log (T)T) and O(log (T)) without and with Slater's condition, respectively. To the best of our knowledge, this paper is the first to achieve tighter (network) cumulative constraint violation bounds for (distributed) online convex optimization with time-varying constraints under Slater's condition. Finally, the theoretical results are verified through numerical simulations. © 1963-2012 IEEE.

关键词： Convex optimization

来源：评论

学校读者我要写书评

暂无评论

Quantized Distributed Nonconvex Optimization with Linear Convergence

Quantized Distributed Nonconvex Optimization with Linear Con...

引用

IEEE Conference on Decision and control

作者： Lei Xu Xinlei Yi Jiayue Sun Yang Shi Karl Henrik Johansson Tao Yang State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang China Division of Decision and Control Systems School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Digital Futures Stockholm Sweden Department of Mechanical Engineering University of Victoria Victoria Canada

ISBN: (数字)9781665467612

ISBN: (纸本)9781665467629

This paper considers distributed nonconvex op-timization for minimizing the average of local cost functions, by using local information exchange over undirected communication networks. Since the communication channels often have limited bandwidth or capacity, we first introduce a quantization rule and an encoder/decoder scheme to reduce the transmission bits. By integrating them with a distributed algorithm, we then propose a distributed quantized nonconvex optimization algorithm. Assuming the global cost function satisfies the Polyak– Łojasiewicz condition, which does not require the global cost function to be convex and the global minimizer is not necessarily unique, we show that the proposed algorithm linearly converges to a global optimal point. Moreover, a low data rate is shown to be sufficient to ensure linear convergence when the algorithm parameters are properly chosen. The theoretical results are illustrated by numerical simulation examples.

关键词： Quantization (signal) Communication channels Bandwidth Cost function Numerical simulation Communication networks Distributed algorithms

来源：评论

学校读者我要写书评

暂无评论

Constrained Optimization with Decision-Dependent Distributions

引用

IEEE Transactions on Automatic control 2025年

作者： Wang, Zifan Liu, Changxin Parisini, Thomas Zavlanos, Michael M. Johansson, Karl H. KTH Royal Institute of Technology Division of Decision and Control Systems School of Electrical Engineering and Computer Science Sweden Digital Futures StockholmSE-10044 Sweden East China University of Science and Technology Key Laboratory of Smart Manufacturing in Energy Chemical Process Ministry of Education Shanghai200237 China Imperial College London Department of Electrical and Electronic Engineering LondonSW7 2AZ United Kingdom Aalborg University Department of Electronic Systems Denmark University of Trieste Department of Engineering and Architecture Italy Duke University Department of Mechanical Engineering and Materials Science DurhamNC United States

In this paper we deal with stochastic optimization problems where the data distributions change in response to the decision variables. Traditionally, the study of optimization problems with decision-dependent distributions has assumed either the absence of constraints or fixed constraints. This work considers a more general setting where the constraints can also dynamically adjust in response to changes in the decision variables. Specifically, we consider linear constraints and analyze the effect of decision-dependent distributions in both the objective function and constraints. Firstly, we establish a sufficient condition for the existence of a constrained equilibrium point, at which the distributions remain invariant under retraining. Moreover, we propose and analyze two algorithms: repeated constrained optimization and repeated dual ascent. For each algorithm, we provide sufficient conditions for convergence to the constrained equilibrium point. Furthermore, we explore the relationship between the equilibrium point and the optimal point for the constrained decision-dependent optimization problem. Notably, our results encompass previous findings as special cases when the constraints remain fixed. To show the effectiveness of our theoretical analysis, we provide numerical experiments on both a market problem and a dynamic pricing problem for parking based on real-world data. © 1963-2012 IEEE.

关键词： Optimization algorithms

来源：评论

学校读者我要写书评

暂无评论

Distributed Constrained Online Nonconvex Optimization with Compressed Communication

arXiv

引用

arXiv 2025年

作者： Zhang, Kunpeng Xu, Lei Yi, Xinlei Cao, Ming Johansson, Karl H. Chai, Tianyou Yang, Tao State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang110819 China Department of Mechanical Engineering University of Victoria VictoriaBCV8W 2Y2 Canada Shanghai Institute of Intelligent Science and Technology National Key Laboratory of Autonomous Intelligent Unmanned Systems Frontiers Science Center for Intelligent Autonomous Systems Ministry of Education Shanghai201210 China Engineering and Technology Institute Groningen Faculty of Science and Engineering University of Groningen GroningenAG 9747 Netherlands Division of Decision and Control Systems School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Sweden Digital Futures Stockholm10044 Sweden

This paper considers distributed online nonconvex optimization with time-varying inequality constraints over a network of agents. For a time-varying graph, we propose a distributed online primal–dual algorithm with compressed communication to efficiently utilize communication resources. We show that the proposed algorithm establishes an O(T max{1−θ1,θ1}) network regret bound and an O(T 1−θ1/2) network cumulative constraint violation bound, where T is the number of iterations and θ1 ∈ (0, 1) is a user-defined trade-off parameter. When Slater’s condition holds (i.e, there is a point that strictly satisfies the inequality constraints at all iterations), the network cumulative constraint violation bound is reduced to O(T 1−θ1). These bounds are comparable to the state-of-the-art results established by existing distributed online algorithms with perfect communication for distributed online convex optimization with (time-varying) inequality constraints. Finally, a simulation example is presented to validate the theoretical results. Copyright © 2025, The Authors. All rights reserved.

关键词： Convex optimization

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：