We revisit the proofs of convergence for a firstorder primal-dual algorithm for convex optimization which we have studied a few years ago. In particular, we prove rates of convergence for a more general version, with...
详细信息
We revisit the proofs of convergence for a firstorder primal-dual algorithm for convex optimization which we have studied a few years ago. In particular, we prove rates of convergence for a more general version, with simpler proofs and more complete results. The new results can deal with explicit terms and nonlinear proximity operators in spaces with quite general norms.
We discuss the foundational role of the proximal framework in the development and analysis of some iconic firstorder optimization algorithms, with a focus on non-Euclidean proximal distances of Bregman type, which ar...
详细信息
We discuss the foundational role of the proximal framework in the development and analysis of some iconic firstorder optimization algorithms, with a focus on non-Euclidean proximal distances of Bregman type, which are central to the analysis of many other fundamental firstorder minimization relatives. We stress simplification and unification by highlighting self-contained elementary proof-patterns to obtain convergence rate and global convergence both in the convex and the nonconvex settings, which in turn also allows to present some novel results.
We use a model LASSO problem to analyze the convergence behavior of the ISTA and FISTA iterations, showing that both iterations satisfy local linear convergence rate bound when close enough to the solution. Using the ...
详细信息
We use a model LASSO problem to analyze the convergence behavior of the ISTA and FISTA iterations, showing that both iterations satisfy local linear convergence rate bound when close enough to the solution. Using the observation that FISTA is an accelerated ISTA process, and a spectral analysis of the associated matrix operators, we show that FISTA's convergence rate can slow down as it proceeds, eventually becoming slower than ISTA. This observation leads to a proposed heuristic algorithm to take an ISTA step if it shows more progress compared to FISTA, as measured by the decrease in the objective function. We illustrate the results with some synthetic numerical examples.
暂无评论