检索结果-内蒙古大学图书馆

A general double-proximal gradient algorithm for d.c. programming

MATHEMATICAL PROGRAMMING 2019年第1-2期178卷 301-326页

作者： Banert, Sebastian Bot, Radu Ioan KTH Royal Inst Technol Dept Math Lindstedtsvagen 25 S-10044 Stockholm Sweden Univ Vienna Fac Math Oskar Morgenstern Pl 1 A-1090 Vienna Austria

The possibilities of exploiting the special structure of d.c. programs, which consist of optimising the difference of convex functions, are currently more or less limited to variants of the DCA proposed by Pham Dinh Tao and Le Thi Hoai An in 1997. These assume that either the convex or the concave part, or both, are evaluated by one of their subgradients. In this paper we propose an algorithm which allows the evaluation of both the concave and the convex part by their proximal points. Additionally, we allow a smooth part, which is evaluated via its gradient. In the spirit of primal-dual splitting algorithms, the concave part might be the composition of a concave function with a linear operator, which are, however, evaluated separately. For this algorithm we show that every cluster point is a solution of the optimisation problem. Furthermore, we show the connection to the Toland dual problem and prove a descent property for the objective function values of a primal-dual formulation of the problem. Convergence of the iterates is shown if this objective function satisfies the Kurdyka-Lojasiewicz property. In the last part, we apply the algorithm to an image processing model.

关键词： d.c. programming Toland dual proximal-gradient algorithm Kurdyka-Lojasiewicz property Convergence analysis

来源：评论

学校读者我要写书评

暂无评论

Distributed proximal-gradient algorithms for nonsmooth convex optimization of second-order multiagent systems

引用

INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL 2020年第17期30卷 7574-7592页

作者： Wang, Qing Chen, Jie Zeng, Xianlin Xin, Bin Beijing Inst Technol Sch Automat 5 South Zhongguancun St Beijing 100081 Peoples R China Beijing Inst Technol State Key Lab Intelligent Control & Decis Complex Beijing Peoples R China Tongji Univ Dept Control Sci & Engn Shanghai Peoples R China Beijing Inst Technol Beijing Adv Innovat Ctr Intelligent Robots & Syst Beijing Peoples R China

This article studies the distributed nonsmooth convex optimization problems for second-order multiagent systems. The objective function is the summation of local cost functions which are convex but nonsmooth. Each agent only knows its local cost function, local constraint set, and neighbor information. By virtue of proximal operator and Lagrangian methods, novel continuous-time distributed proximal-gradient algorithms with derivative feedback are proposed to solve the nonsmooth convex optimization for the consensus and resource allocation of multiagent systems, respectively. With the proposed algorithms, both the consensus and resource allocation problems are solved. Moreover, the system can converge to the optimal solution. Finally, simulation examples are given to illustrate the effectiveness of the proposed algorithms.

关键词： multiagent systems nonsmooth convex optimization optimal consensus proximal-gradient algorithm resource allocation

来源：评论

学校读者我要写书评

暂无评论

Stochastic proximal-gradient algorithms for penalized mixed models

引用

STATISTICS AND COMPUTING 2019年第2期29卷 231-253页

作者： Fort, Gersende Ollier, Edouard Samson, Adeline Univ Toulouse CNRS IMT UMR5219 F-31062 Toulouse 9 France INSERM U1059 Dysfonct Vasc & Hemostase St Etienne France Ecole Normale Super Lyon UMPA CNRS INRIAProject Team NUMEDUMR 5669 46 Allee Italie F-69364 Lyon 07 France Univ Grenoble Alpes CNRS Lab Jean Kuntzmann UMR 5224 F-38000 Grenoble France

Motivated by penalized likelihood maximization in complex models, we study optimization problems where neither the function to optimize nor its gradient has an explicit expression, but its gradient can be approximated by a Monte Carlo technique. We propose a new algorithm based on a stochastic approximation of the proximal-gradient (PG) algorithm. This new algorithm, named stochastic approximation PG (SAPG) is the combination of a stochastic gradient descent step whichroughly speakingcomputes a smoothed approximation of the gradient along the iterations, and a proximal step. The choice of the step size and of the Monte Carlo batch size for the stochastic gradient descent step in SAPG is discussed. Our convergence results cover the cases of biased and unbiased Monte Carlo approximations. While the convergence analysis of some classical Monte Carlo approximation of the gradient is already addressed in the literature (see Atchade etal. in J Mach Learn Res 18(10):1-33, 2017), the convergence analysis of SAPG is new. Practical implementation is discussed, and guidelines to tune the algorithm are given. The two algorithms are compared on a linear mixed effect model as a toy example. A more challenging application is proposed on nonlinear mixed effect models in high dimension with a pharmacokinetic data set including genomic covariates. To our best knowledge, our work provides the first convergence result of a numerical method designed to solve penalized maximum likelihood in a nonlinear mixed effect model.

关键词： proximal-gradient algorithm Stochastic gradient Stochastic EM algorithm Stochastic approximation Nonlinear mixed effect models

来源：评论

学校读者我要写书评

暂无评论

A distributed feasible-side convergent modifier-adaptation scheme for interconnected systems, with application to gas-compressor stations

引用

COMPUTERS & CHEMICAL ENGINEERING 2018年 115卷 474-486页

作者： Milosavljevic, Predrag Schneider, Rene Cortinovis, Andrea Faulwasser, Timm Bonvin, Dominique Ecole Polytech Fed Lausanne Lab Automat CH-1015 Lausanne Switzerland Karlsruhe Inst Technol Inst Automat & Appl Informat D-76344 Eggenstein Leopoldshafen Germany ABB Switzerland Ltd Corp Res CH-5405 Baden Switzerland

Recently, a feasible-side globally convergent modifier-adaptation scheme has been proposed for the real-time optimization of uncertain systems. We show that this scheme is related to proximal-gradient algorithms in numerical optimization and we exploit this relationship to analyze its convergence in the case of inexact gradient information. We also make use of this relationship to propose a novel distributed modifier-adaptation algorithm for interconnected systems that uses a coordinator and knowledge of the interconnection topology. We then prove its feasible-side convergence to plant optimality. In addition, our distributed algorithm ensures confidentiality of local models and data. We finally demonstrate the applicability and effectiveness of this algorithm on a load-sharing optimization case study for serially connected gas compressors. (C) 2018 Elsevier Ltd. All rights reserved.

关键词： Real-time optimization Interconnected systems Modifier adaptation Model uncertainty proximal-gradient algorithm Gas compressors

来源：评论

学校读者我要写书评

暂无评论

Convergence of Inertial Dynamics Driven by Sums of Potential and Nonpotential Operators with Implicit Newton-Like Damping

引用

JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS 2023年第1期198卷 290-331页

作者： Adly, Samir Attouch, Hedy Vo, Van Nam Univ Limoges Lab XLIM 123 Ave A Thomas F-87060 Limoges France Univ Montpellier IMAG CNRS Montpellier France

We analyze the convergence properties when the time t tends to infinity of the trajectories generated by damped inertial dynamics which are driven by the sum of potential and nonpotential operators. Precisely, we seek to reach asymptotically the zeros of a maximally monotone operator which is the sum of a potential operator (the gradient of a continuously differentiable convex function) and of a monotone and cocoercive nonpotential operator. As an original feature, in addition to the viscous friction, the dynamic involves implicit Newton-type damping. This contrasts with the authors' previous study where explicit Newton-type damping was considered, which, for the potential term corresponds to Hessian-driven damping. We show the weak convergence, as time tends to infinity, of the generated trajectories toward the zeros of the sum of the potential and nonpotential operators. Our results are based on Lyapunov analysis and appropriate setting of the damping parameters. The introduction of geometric dampings allows to control and attenuate the oscillations known for the viscous damping of inertial methods. Rewriting the second-order evolution equation as a system involving only first-order derivative in time and space allows us to extend the convergence analysis to nonsmooth convex potentials. The main part of our study concerns the autonomous case with positive fixed parameters. We complete it with some first results concerning the nonautonomous case, and which are based on a recent acceleration method using time scaling and averaging. These results open the door to the design of new first-order accelerated algorithms in optimization taking into account the specific properties of potential and nonpotential terms. The proofs and techniques are original due to the presence of the nonpotential term.

关键词： proximal-gradient algorithm Inertial method Hessian-driven damping Nonpotential operator Cocoercive operator Structured monotone equation Implicit Newton-like damping

来源：评论

学校读者我要写书评

暂无评论

ACCELERATED FORWARD-BACKWARD OPTIMIZATION USING DEEP LEARNING

引用

SIAM JOURNAL ON OPTIMIZATION 2024年第2期34卷 1236-1263页

作者： Banert, Sebastian Rudzusika, Jevgenija Oktem, Ozan Adler, Jonas Lund Univ Dept Automat Control S-22363 Lund Sweden KTH Royal Inst Technol Dept Math S-10044 Stockholm Sweden DeepMind London England

We propose several deep -learning accelerated optimization solvers with convergence guarantees. We use ideas from the analysis of accelerated forward -backward schemes like FISTA, but instead of the classical approach of proving convergence for a choice of parameters, such as a step -size, we show convergence whenever the update is chosen in a specific set. Rather than picking a point in this set using some predefined method, we train a deep neural network to pick the best update within a given space. Finally, we show that the method is applicable to several cases of smooth and nonsmooth optimization and show superior results to established accelerated solvers.

关键词： convex optimization deep learning proximal-gradient algorithm inverse problems

来源：评论

学校读者我要写书评

暂无评论

Newton-Type Inertial algorithms for Solving Monotone Equations Governed by Sums of Potential and Nonpotential Operators

引用

APPLIED MATHEMATICS AND OPTIMIZATION 2022年第3期85卷 1-33页

作者： Adly, Samir Attouch, Hedy Van Nam Vo Univ Limoges Lab XLIM 123 Ave Albert Thomas F-87060 Limoges France Univ Montpellier CNRS IMAG Pl Eugene Bataillon F-34095 Montpellier 5 France

In a Hilbert space setting, this paper is devoted to the study of a class of first-order algorithms which aim to solve structured monotone equations involving the sum of potential and nonpotential operators. Precisely, we are looking for the zeros of an operator A = del f + B, where del f is the gradient of a differentiable convex function f, and B is a nonpotential monotone and cocoercive operator. This study is based on the inertial autonomous dynamic previously studied by the authors, which involves dampings controlled respectively by the Hessian of f, and by a Newton-type correction term attached to B. These geometric dampings attenuate the oscillations which occur with the inertial methods with viscous damping. Temporal discretization of this dynamic provides fully splitted proximal-gradient algorithms. Their convergence properties are proved using Lyapunov analysis. These results open the door to the design of first-order accelerated algorithms in numerical optimization taking into account the specific properties of potential and nonpotential terms.

关键词： proximal-gradient algorithm Inertial method Hessian-driven damping Nonpotential operator Cocoercive operator Structured monotone equation

来源：评论

学校读者我要写书评

暂无评论

COPOSITIVITY MEETS D. C. OPTIMIZATION

引用

JOURNAL OF DYNAMICS AND GAMES 2022年第1期9卷 27-32页

作者： Moudafi, Abdellatif Mainge, Paul-Emile Aix Marseille Univ CNRS UMR 7020 AMUUTLNLab Informat & Syst LIS Marseille France Univ Antilles F-97233 Martinique France

Based on a work by M. Dur and J.-B. Hiriart-Urruty[3], we consider the problem of whether a symmetric matrix is copositive formulated as a difference of convex functions problem. The convex nondifferentiable function in this d.c. decomposition being proximable, we then apply a proximal-gradient method to approximate the related stationary points. Whereas, in [3], the DCA algorithm was used.

关键词： Copositive matrices difference of convex functions proximal-gradient algorithm DC optimization critical point

来源：评论

学校读者我要写书评

暂无评论

An Inertial proximal-gradient Penalization Scheme for Constrained Convex Optimization Problems

引用

VIETNAM JOURNAL OF MATHEMATICS 2018年第1期46卷 53-71页

作者： Bot, Radu Ioan Csetnek, Erno Robert Nimana, Nimit Univ Vienna Fac Math Oskar Morgenstern Pl 1 A-1090 Vienna Austria Naresuan Univ Dept Math Fac Sci Phitsanulok 65000 Thailand

We propose a proximal-gradient algorithm with penalization terms and inertial and memory effects for minimizing the sum of a proper, convex, and lower semicontinuous and a convex differentiable function subject to the set of minimizers of another convex differentiable function. We show that, under suitable choices for the step sizes and the penalization parameters, the generated iterates weakly converge to an optimal solution of the addressed bilevel optimization problem, while the objective function values converge to its optimal objective value.

关键词： proximal-gradient algorithm Inertial algorithm Penalization Fenchel conjugate

来源：评论

学校读者我要写书评

暂无评论

On critical points of the difference of two maximal monotone operators

引用

AFRIKA MATEMATIKA 2015年第3-4期26卷 457-463页

作者： Moudafi, A. Univ Antilles Guyane Ceregmia Dept Sci Interfac F-97200 Schoelcher Martinique France

Relying on the Yosida approximate, we present an algorithmic approach for finding a critical point of the difference of two maximal monotone operators in a finite dimensional real Hilbert space. The problem can be seen as a new generalization of the DC programming case. The latter is also studied and a convergence result is provided.

关键词： Maximal monotone operators proximal-gradient algorithm Regularization DC programming

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：