检索结果-内蒙古大学图书馆

DISTRIBUTED proximal-gradient method FOR CONVEX OPTIMIZATION WITH INEQUALITY CONSTRAINTS

ANZIAM JOURNAL 2014年第2期56卷 160-178页

作者： Li, Jueyou Wu, Changzhi Wu, Zhiyou Long, Qiang Wang, Xiangyu Chongqing Normal Univ Sch Math Chongqing 400047 Peoples R China Federat Univ Australia Sch SITE Ballarat Vic 3353 Australia Curtin Univ Sch Built Environm Australasian Joint Res Ctr Bldg Informat Modellin Bentley WA 6102 Australia Southwest Univ Sci & Technol Sch Sci Chengdu 621010 Sichuan Peoples R China Kyung Hee Univ Dept Housing & Interior Design Seoul South Korea

We consider a distributed optimization problem over a multi-agent network, in which the sum of several local convex objective functions is minimized subject to global convex inequality constraints. We first transform the constrained optimization problem to an unconstrained one, using the exact penalty function method. Our transformed problem has a smaller number of variables and a simpler structure than the existing distributed primal-dual subgradient methods for constrained distributed optimization problems. Using the special structure of this problem, we then propose a distributed proximal-gradient algorithm over a time-changing connectivity network, and establish a convergence rate depending on the number of iterations, the network topology and the number of agents. Although the transformed problem is nonsmooth by nature, our method can still achieve a convergence rate, O(1/k), after k iterations, which is faster than the rate, O(1/root k), of existing distributed subgradient-based methods. Simulation experiments on a distributed state estimation problem illustrate the excellent performance of our proposed method.

关键词： distributed algorithm proximal-gradient method exact penalty function method convex optimization

来源：评论

学校读者我要写书评

暂无评论

A Distributed Stochastic proximal-gradient Algorithm for Composite Optimization

引用

IEEE TRANSACTIONS ON CONTROL OF NETWORK SYSTEMS 2021年第3期8卷 1383-1393页

作者： Niu, Youcheng Li, Huaqing Wang, Zheng Lu, Qingguo Xia, Dawen Ji, Lianghao Southwest Univ Coll Elect & Informat Engn Chongqing Key Lab Nonlinear Circuits & Intelligen Chongqing 400715 Peoples R China Univ New South Wales Sch Elect Engn & Telecommun Sydney NSW 2052 Australia Guizhou Minzu Univ Coll Data Sci & Informat Engn Guiyang 550025 Peoples R China Chongqing Univ Posts & Telecommun Chongqing Key Lab Image Cognit Chongqing 400065 Peoples R China

In this article, we consider distributed composite optimization problems involving a common non-smooth regularization term over an undirected and connected network. Inspired by vast applications of this kind of problem in large-scale machine learning, the local cost function of each node is further set as an average of a certain amount of local cost subfunctions. For this scenario, most existing solutions based on the proximal method tend to ignore the cost of gradient evaluations, which results in degraded performance. We instead develop a distributed stochastic proximal-gradient algorithm to tackle the problems by employing the local unbiased stochastic averaging gradient method. At each iteration, only a single local cost subfunction is demanded by each node to evaluate the gradient, then the average of the latest stochastic gradients serves as the approximation of the true local gradient. An explicit linear convergence rate of the proposed algorithm is established with constant dual step-sizes for strongly convex local cost subfunctions with Lipschitz-continuous gradients. Furthermore, it is shown that, in the smooth cases, our simplified analysis technique can be extended to some notable primal-dual domain algorithms, such as DSA, EXTRA, and DIGing. Numerical experiments are presented to confirm the theoretical findings.

关键词： Linear convergence linear convergence distributed composite optimization machine learning proximal-gradient method stochastic averaging gradient (SAG)

来源：评论

学校读者我要写书评

暂无评论

Fast reconstruction algorithm for perturbed compressive sensing based on total least-squares and proximal splitting

引用

SIGNAL PROCESSING 2017年第0期130卷 57-63页

作者： Arablouei, Reza CSIRO Pullenvale Qld Australia

We consider the problem of finding a sparse solution for an underdetermined linear system of equations when the known parameters on both sides of the system are subject to perturbation. This problem is particularly relevant to reconstruction in fully-perturbed compressive-sensing setups where both the projected measurements of an unknown sparse vector and the knowledge of the associated projection matrix are perturbed due to noise, error, mismatch, etc. We propose a new iterative algorithm for tackling this problem. The proposed algorithm utilizes the proximal-gradient method to find a sparse total least-squares solution by minimizing an l(1)-regularized Rayleigh-quotient cost function. We determine the step-size of the algorithm at each iteration using an adaptive rule accompanied by backtracking line search to improve the algorithm's convergence speed and preserve its stability. The proposed algorithm is considerably faster than a popular previously-proposed algorithm, which employs the alternating-direction method and coordinate-descent iterations, as it requires significantly fewer computations to deliver the same accuracy. We demonstrate the effectiveness of the proposed algorithm via simulation results. (C) 2016 Elsevier B.V. All rights reserved.

关键词： Non-convex optimization Perturbed compressive sensing proximal-gradient method Rayleigh quotient Sparse reconstruction Sparse total least-squares

来源：评论

学校读者我要写书评

暂无评论

An Edge-based Stochastic proximal gradient Algorithm for Decentralized Composite Optimization

引用

INTERNATIONAL JOURNAL OF CONTROL AUTOMATION AND SYSTEMS 2021年第11期19卷 3598-3610页

作者： Zhang, Ling Yan, Yu Wang, Zheng Li, Huaqing Southwest Univ Coll Elect & Informat Engn Chongqing Key Lab Nonlinear Circuits & Intelligen Chongqing 400715 Peoples R China Univ New South Wales Sch Elect Engn & Telecommun Sydney NSW 2052 Australia

This paper investigates decentralized composite optimization problems involving a common non-smooth regularization term over an undirected and connected network. In the same situation, there exist lots of gradient-based proximal distributed methods, but most of them are only sublinearly convergent. The proof of linear convergence for this series of algorithms is extremely difficult. To set up the problem, we presume all networked agents use the same non-smooth regularization term, which is the circumstance for most machine learning to implement based on centralized optimization. For this scenario, most existing proximal-gradient algorithms trend to ignore the cost of gradient evaluations, which results in degraded performance. To tackle this problem, we further set the local cost function to the average of a moderate amount of local cost subfunctions and develop an edge-based stochastic proximal gradient algorithm (SPG-Edge) by employing local unbiased stochastic averaging gradient method. When the non-smooth term does not exist, the proposed algorithm could be extended to some notable primal-dual domain algorithms, such as EXTRA and DIGing. Finally, we provide a simplified proof of linear convergence and conduct numerical experiments to illustrate the validity of theoretical results.

关键词： Decentralized composite optimization linear convergence machine learning proximal-gradient method stochastic averaging gradient

来源：评论

学校读者我要写书评

暂无评论

Dynamical systems and forward-backward algorithms associated with the sum of a convex subdifferential and a monotone cocoercive operator

引用

OPTIMIZATION 2015年第10期64卷 2223-2252页

作者： Abbas, Boushra Attouch, Hedy Univ Montpellier 2 Inst Math & Modelisat Montpellier UMR CNRS 5149 F-34095 Montpellier 5 France

In a Hilbert framework, we introduce continuous and discrete dynamical systems which aim at solving inclusions governed by structured monotone operators , where is the subdifferential of a convex lower semicontinuous function , and is a monotone cocoercive operator. We first consider the extension to this setting of the regularized Newton dynamic with two potentials which was considered in Abbas, Attouch, Svaiter JOTA, 2014. Then, we revisit some related dynamical systems, namely the semigroup of contractions generated by , and the continuous gradient projection dynamic. By a Lyapunov analysis, we show the convergence properties of the orbits of these systems, thereby extending the known results. The time discretization of these dynamics gives various forward-backward splitting methods (some new) for solving structured monotone inclusions involving non-potential terms. The convergence of these algorithms is obtained under classical step size limitation. Perspectives are given in the field of numerical splitting methods for optimization, and multicriteria decision processes.

关键词： 47J35 49M37 47J30 90C25 65K15 49M15 34G25 90C53 47J25 forward-backward algorithms structured monotone inclusions cocoercive operators Lyapunov analysis weak asymptotic convergence multiobjective decision dissipative dynamics proximal-gradient method subdifferential operators Levenberg-Marquardt regularization

来源：评论

学校读者我要写书评

暂无评论

Backward-forward algorithms for structured monotone inclusions in Hilbert spaces

引用

JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS 2018年第2期457卷 1095-1117页

作者： Attouch, Hedy Peypouquet, Juan Redont, Patrick Univ Montpellier 2 Inst Montpellierain Alexander Grothendieek IMAG UMR CNRS 5149 Pl Eugene Bataillon F-34095 Montpellier 5 France Univ Tecn Federico Santa Maria Dept Matemat Ave Espana 1680 Valparaiso Chile

In this paper, we study the backward forward algorithm as a splitting method to solve structured monotone inclusions, and convex minimization problems in Hilbert spaces. It has a natural link with the forward backward algorithm and has the same computational complexity, since it involves the same basic blocks, but organized differently. Surprisingly enough, this kind of iteration arises when studying the time discretization of the regularized Newton method for maximally monotone operators. First, we show that these two methods enjoy remarkable involutive relations, which go far beyond the evident inversion of the order in which the forward and backward steps are applied. Next, we establish several convergence properties for both methods, some of which were unknown even for the forward backward algorithm. This brings further insight into this well-known scheme. Finally, we specialize our results to structured convex minimization problems, the gradient projection algorithms, and give a numerical illustration of theoretical interest. (C) 2016 Elsevier Inc. All rights reserved.

关键词： Monotone inclusion Forward backward algorithm proximal-gradient method

来源：评论

学校读者我要写书评

暂无评论

SIGNPROX: ONE-BIT proximal ALGORITHM FOR NONCONVEX STOCHASTIC OPTIMIZATION 44

SIGNPROX: ONE-BIT PROXIMAL ALGORITHM FOR NONCONVEX STOCHASTI...

引用

44th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

作者： Xu, Xiaojian Kamilov, Ulugbek S. Washington Univ Dept Comp Sci & Engn St Louis MO 63130 USA Washington Univ Dept Elect & Syst Engn St Louis MO 63130 USA

ISBN: (纸本)9781479981311

Stochastic gradient descent (SGD) is one of the most widely used optimization methods for parallel and distributed processing of large datasets. One of the key limitations of distributed SGD is the need to regularly communicate the gradients between different computation nodes. To reduce this communication bottleneck, recent work has considered a one-bit variant of SGD, where only the sign of each gradient element is used in optimization. In this paper, we extend this idea by proposing a stochastic variant of the proximal-gradient method that also uses one-bit per update element. We prove the theoretical convergence of the method for non-convex optimization under a set of explicit assumptions. Our results indicate that the compressed method can match the convergence rate of the uncompressed one, making the proposed method potentially appealing for distributed processing of large datasets.

关键词： proximal-gradient method forward-backward algorithm stochastic gradient descent nonconvex optimization

来源：评论

学校读者我要写书评

暂无评论

REGULARIZED FOURIER PTYCHOGRAPHY USING AN ONLINE PLUG-AND-PLAY ALGORITHM 44

REGULARIZED FOURIER PTYCHOGRAPHY USING AN ONLINE PLUG-AND-PL...

引用

44th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

作者： Sun, Yu Xu, Shiqi Li, Yunzhe Tian, Lei Wohlberg, Brendt Kamilov, Ulugbek S. Washington Univ Dept Comp Sci & Engn St Louis MO 63130 USA Washington Univ Dept Elect & Syst Engn St Louis MO 63130 USA Boston Univ Dept Elect & Comp Engn Boston MA 02215 USA Los Alamos Natl Lab Div Theoret Los Alamos NM 87545 USA

ISBN: (纸本)9781479981311

The plug-and-play priors (PnP) framework has been recently shown to achieve state-of-the-art results in regularized image reconstruction by leveraging a sophisticated denoiser within an iterative algorithm. In this paper, we propose a new online PnP algorithm for Fourier ptychographic microscopy (FPM) based on the accelerated proximal gradient method (APGM). Specifically, the proposed algorithm uses only a subset of measurements, which makes it scalable to a large set of measurements. We validate the algorithm by showing that it can lead to significant performance gains on both simulated and experimental data.

关键词： Fourier ptychography plug-and-play priors stochastic gradient descent proximal-gradient method forward-backward algorithm nonconvex optimization

来源：评论

学校读者我要写书评

暂无评论

Integer optimal control problems with total variation regularization: Optimality conditions and fast solution of subproblems

引用

ESAIM-CONTROL OPTIMISATION AND CALCULUS OF VARIATIONS 2023年 29卷

作者： Marko, Jonas Wachsmuth, Gerd Brandenburg Tech Univ Cottbus Senftenberg Inst Math D-03046 Cottbus Germany

We investigate local optimality conditions of first and second order for integer optimal control problems with total variation regularization via a finite-dimensional switching-point problem. We show the equivalence of local optimality for both problems, which will be used to derive conditions concerning the switching points of the control function. A non-local optimality condition treating back-and-forth switches will be formulated. For the numerical solution, we propose a proximal-gradient method. The emerging discretized subproblems will be solved by employing Bellman's optimality principle, leading to an algorithm which is polynomial in the mesh size and in the admissible control levels. An adaption of this algorithm can be used to handle subproblems of the trust-region method proposed in Leyffer and Manns, ESAIM: Control Optim. Calc. Var. 28 (2022) 66. Finally, we demonstrate computational results.

关键词： Integer optimal control problem total variation regularization switching-point optimization proximal-gradient method trust-region method

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：