This paper reveals that a common and central role, played in many error bound (EB) conditions and a variety of gradient-type methods, is a residual measure operator. On one hand, by linking this operator with other op...
详细信息
This paper reveals that a common and central role, played in many error bound (EB) conditions and a variety of gradient-type methods, is a residual measure operator. On one hand, by linking this operator with other optimality measures, we define a group of abstract EB conditions, and then analyze the interplay between them;on the other hand, by using this operator as an ascent direction, we propose an abstract gradient-type method, and then derive EB conditions that are necessary and sufficient for its linear convergence. The former provides a unified framework that not only allows us to find new connections between many existing EB conditions, but also paves a way to construct new ones. The latter allows us to claim the weakest conditions guaranteeing linear convergence for a number of fundamental algorithms, including the gradient method, the proximal point algorithm, and the forward-backward splitting algorithm. In addition, we show linear convergence for the proximal alternating linearized minimization algorithm under a group of equivalent EB conditions, which are strictly weaker than the traditional strongly convex condition. Moreover, by defining a new EB condition, we show Q-linear convergence of Nesterov's accelerated forward-backwardalgorithm without strong convexity. Finally, we verify EB conditions for a class of dual objective functions.
We study the extension of the proximal gradient algorithm where only a stochastic gradient estimate is available and a relaxation step is allowed. We establish convergence rates for function values in the convex case,...
详细信息
We study the extension of the proximal gradient algorithm where only a stochastic gradient estimate is available and a relaxation step is allowed. We establish convergence rates for function values in the convex case, as well as almost sure convergence and convergence rates for the iterates under further convexity assumptions. Our analysis avoid averaging the iterates and error summability assumptions which might not be satisfied in applications, e.g. in machine learning. Our proofing technique extends classical ideas from the analysis of deterministic proximal gradient algorithms.
We propose and analyze the convergence of a novel stochastic algorithm for monotone inclusions that are sum of a maximal monotone operator and a single-valued cocoercive operator. The algorithm we propose is a natural...
详细信息
We propose and analyze the convergence of a novel stochastic algorithm for monotone inclusions that are sum of a maximal monotone operator and a single-valued cocoercive operator. The algorithm we propose is a natural stochastic extension of the classical forward-backward method. We provide a non-asymptotic error analysis in expectation for the strongly monotone case, as well as almost sure convergence under weaker assumptions. For minimization problems, we recover rates matching those obtained by stochastic extensions of the so-called accelerated methods. Stochastic quasi-Fejer's sequences are a key technical tool to prove almost sure convergence.
We propose a variable metric forward-backward splitting algorithm and prove its convergence in real Hilbert spaces. We then use this framework to derive primal-dual splittingalgorithms for solving various classes of ...
详细信息
We propose a variable metric forward-backward splitting algorithm and prove its convergence in real Hilbert spaces. We then use this framework to derive primal-dual splittingalgorithms for solving various classes of monotone inclusions in duality. Some of these algorithms are new even when specialized to the fixed metric case. Various applications are discussed.
This paper deals with the quadrature amplitude modulation (QAM) problem for the multiple-input multiple-output (MIMO) channel. Based on the maximum likelihood estimation, the QAM detection problem is formulated as an ...
详细信息
ISBN:
(纸本)9781479952557
This paper deals with the quadrature amplitude modulation (QAM) problem for the multiple-input multiple-output (MIMO) channel. Based on the maximum likelihood estimation, the QAM detection problem is formulated as an integer quadratic programming, which is a combinatorial problem and difficult to obtain exact solutions. In order to overcome combinatorial difficulties, this paper formulates the QAM detection problem as the l(o) norm minimization problem and relaxes it into a quadratic programming with the l(1) norm regularization. Utilizing and modifying the forward-backwardsplitting (FOBOS) algorithm, a new QAM detection algorithm is proposed. This algorithm has a trade-off between the computational cost and the detection accuracy, which depends on a parameter of the algorithm. Numerical simulations show that the proposed algorithm works well and achieves a good detection performance with less computational cost comparing with the semidefinite relaxation (SDR) based algorithm.
In this paper, we apply compressed sensing (CS) to video compression. CS techniques exploit the observation that one needs much fewer random measurements than given by the Shannon-Nyquist sampling theory to recover an...
详细信息
In this paper, we apply compressed sensing (CS) to video compression. CS techniques exploit the observation that one needs much fewer random measurements than given by the Shannon-Nyquist sampling theory to recover an object if this object is compressible (i.e., sparse in the spatial domain or in a transform domain). In the CS framework, we can achieve sensing, compression, and denoising simultaneously. We propose a fast and simple online encoding by the application of pseudorandom downsampling of the 2-D fast Fourier transform to video frames. For offline decoding, we apply a modification of the recently proposed approximate message passing (AMP) algorithm. The AMP method has been derived using the statistical concept of "state evolution," and it has been shown to considerably accelerate the convergence rate in special CS-decoding applications. We shall prove that the AMP method can be rewritten as a forward-backward splitting algorithm. This new representation enables us to give conditions that ensure convergence of the AMP method and to modify the algorithm in order to achieve higher robustness. The success of reconstruction methods for video decoding also essentially depends on the chosen transform, where sparsity of the video signals is assumed. We propose incorporating the 3-D dual-tree complex wavelet transform that possesses sufficiently good directional selectivity while being computationally less expensive and less redundant than other directional 3-D wavelet transforms.
We derive strongly convergent algorithms to solve inverse problems involving elastic-net regularization. Moreover, using functional analysis techniques, we provide a rigorous study of the asymptotic properties of the ...
详细信息
We derive strongly convergent algorithms to solve inverse problems involving elastic-net regularization. Moreover, using functional analysis techniques, we provide a rigorous study of the asymptotic properties of the regularized solutions that allows to cast in a unified framework 1, elastic-net and classical Tikhonov regularization.
The notion of soft thresholding plays a central role in problems from various areas of applied mathematics, in which the ideal solution is known to possess a sparse decomposition in some orthonormal basis. Using conve...
详细信息
The notion of soft thresholding plays a central role in problems from various areas of applied mathematics, in which the ideal solution is known to possess a sparse decomposition in some orthonormal basis. Using convex-analytical tools, we extend this notion to that of proximal thresholding and investigate its properties, providing, in particular, several characterizations of such thresholders. We then propose a versatile convex variational formulation for optimization over orthonormal bases that covers a wide range of problems, and we establish the strong convergence of a proximal thresholding algorithm to solve it. Numerical applications to signal recovery are demonstrated.
暂无评论