This paper proposes and develops inexact proximal methods for finding stationary points of the sum of a smooth function and a nonsmooth weaklyconvex one, where an error is present in the calculation of the proximal m...
详细信息
This paper proposes and develops inexact proximal methods for finding stationary points of the sum of a smooth function and a nonsmooth weaklyconvex one, where an error is present in the calculation of the proximal mapping of the nonsmooth term. A general framework for finding zeros of a continuous mapping is derived from our previous paper on this subject to establish convergence properties of the inexact proximal point method when the smooth term is vanished and of the inexact proximal gradient method when the smooth term satisfies a descent condition. The inexact proximal point method achieves global convergence with constructive convergence rates when the Moreau envelope of the objective function satisfies the Kurdyka-& Lstrok;ojasiewicz (KL) property. Meanwhile, when the smooth term is twice continuously differentiable with a Lipschitz continuous gradient and a differentiable approximation of the objective function satisfies the KL property, the inexact proximal gradient method achieves the global convergence of iterates with constructive convergence rates.
We investigate proximal epsilon-subdifferentials and derive sum rules that hold for weaklyconvex function, by incorporating the corresponding moduli of weak convexity into the respective formulas. As an application, ...
详细信息
ISBN:
(纸本)9783907144084
We investigate proximal epsilon-subdifferentials and derive sum rules that hold for weaklyconvex function, by incorporating the corresponding moduli of weak convexity into the respective formulas. As an application, we analyse inexact proximity operators for weakly convex functions in terms of proximal e-subdifferentials and the related notion of criticality.
We propose a general scheme for solving convex and non-convex optimization problems on manifolds. The central idea is that, by adding a multiple of the squared retraction distance to the objective function in question...
详细信息
We propose a general scheme for solving convex and non-convex optimization problems on manifolds. The central idea is that, by adding a multiple of the squared retraction distance to the objective function in question, we "convexify" the objective function and solve a series of convex sub-problems in the optimization procedure. Our proposed algorithm adapts to the level of complexity in the objective function without requiring the knowledge of the convexity of non-convexity of the objective function. We show that when the objective function is convex, the algorithm provably converges to the optimum and leads to accelerated convergence. When the objective function is non-convex, the algorithm will converge to a stationary point. Our proposed method unifies insights from Nesterov's original idea for accelerating gradient descent algorithms with recent developments in optimization algorithms in Euclidean space. We demonstrate the utility of our algorithms on several manifold optimization tasks such as estimating intrinsic and extrinsic Fr & eacute;chet means on spheres and low-rank matrix factorization with Grassmann manifolds applied to the Netflix rating data set.
This paper proposes a proximal variant of the alternating direction method of multipliers (ADMM) for distributed optimization. Although the current versions of ADMM algorithm provide promising numerical results in pro...
详细信息
ISBN:
(纸本)9798350300673
This paper proposes a proximal variant of the alternating direction method of multipliers (ADMM) for distributed optimization. Although the current versions of ADMM algorithm provide promising numerical results in producing solutions that are close to optimal for many convex and non-convex optimization problems, it remains unclear if they can converge to a stationary point for weaklyconvex and locally non-smooth functions. Through our analysis using the Moreau envelope function, we demonstrate that MADM can indeed converge to a stationary point under mild conditions. Our analysis also includes computing the bounds on the amount of change in the dual variable update step by relating the gradient of the Moreau envelope function to the proximal function. Furthermore, the results of our numerical experiments indicate that our method is faster and more robust than widely-used approaches.
In this paper, we consider stochastic weaklyconvex optimization problems, however without the existence of a stochastic subgradient oracle. We present a derivative free algorithm that uses a two point approximation f...
详细信息
In this paper, we consider stochastic weaklyconvex optimization problems, however without the existence of a stochastic subgradient oracle. We present a derivative free algorithm that uses a two point approximation for computing a gradient estimate of the smoothed function. We prove convergence at a similar rate as state of the art methods, however with a larger constant, and report some numerical results showing the effectiveness of the approach.
Rockafellar has shown that the subdifferentials of convexfunctions are always cyclically monotone operators. Moreover, maximal cyclically monotone operators are necessarily operators of this type, since one can const...
详细信息
Rockafellar has shown that the subdifferentials of convexfunctions are always cyclically monotone operators. Moreover, maximal cyclically monotone operators are necessarily operators of this type, since one can construct explicitly a convex function, which turns out to be unique up to a constant, whose subdifferential gives back the operator. This result is a cornerstone in convex analysis and relates tightly convexity and monotonicity. In this paper, we establish analogous robust results that relate weak convexity notions to corresponding notions of weak monotonicity, provided one deals with locally Lipschitz functions and locally bounded operators. In particular, the subdifferentials of locally Lipschitz functions that are directionally hypomonotone [ respectively, directionally submonotone] enjoy also an additional cyclic strengthening of this notion and in fact are maximal under this new property. Moreover, every maximal cyclically hypomonotone [ respectively, maximal cyclically submonotone] operator is always the Clarke subdifferential of some directionally weaklyconvex [ respectively, directionally approximately convex] locally Lipschitz function, unique up to a constant, which in finite dimentions is a lower C-2 function [ respectively, a lower C-1 function].
We give a definition of the class of functions with a concave minorant and compare these functions with other classes of functions often used in global optimization, e.g. weakly convex functions, d.c. functions, Lipsc...
详细信息
We give a definition of the class of functions with a concave minorant and compare these functions with other classes of functions often used in global optimization, e.g. weakly convex functions, d.c. functions, Lipschitzian functions, continuous and lower semicontinuous functions. It is shown that the class of functions with a concave minorant is closed under operations mainly used in optimization and how a concave minorant can be constructed for a given function.
暂无评论