Based on the insight gained from the three-term conjugategradient methods suggested by Zhang et al. (Optim Methods Softw 22:697-711, 2007) two nonlinear conjugategradient methods are proposed, making modifications o...
详细信息
Based on the insight gained from the three-term conjugategradient methods suggested by Zhang et al. (Optim Methods Softw 22:697-711, 2007) two nonlinear conjugategradient methods are proposed, making modifications on the conjugategradient methods proposed by Dai and Liao (Appl Math Optim 43:87-101, 2001), and Zhou and Zhang (Optim Methods Softw 21:707-714, 2006). The methods can be regarded as modified versions of two three-term conjugategradient methods proposed by Sugiki et al. (J Optim Theory Appl 153:733-757, 2012) in which the search directions are computed using the secant equations in a way to achieve the sufficient descent property. One of the methods is shown to be globally convergent for uniformly convex objective functions while the other is shown to be globally convergent without convexity assumption on the objective function. Comparative numerical results demonstrating efficiency of the proposed methods are reported.
This paper revisits, in a multi-thread context, the so-called multi-parameter or block conjugategradient (B-CG) methods, first proposed as sequential algorithms by O'Leary and Brezinski, for the solution of the l...
详细信息
This paper revisits, in a multi-thread context, the so-called multi-parameter or block conjugategradient (B-CG) methods, first proposed as sequential algorithms by O'Leary and Brezinski, for the solution of the linear system Ax = b, for an n-dimensional symmetric positive definite matrix A. Instead of the scalar parameters of the classical CG algorithm, which minimizes a scalar functional at each iteration, multiple descent and conjugate directions are updated simultaneously. Implementation involves the use of multiple threads and the algorithm is referred to as cooperative CG (CCG) to emphasize that each thread now uses information that comes from the other threads. It is shown that for a sufficiently large matrix dimension n, the use of an optimal number of threads results in a worst case flop count of O (n(7/3)) in exact arithmetic. Numerical experiments on a multi-core, multi-thread computer, for synthetic and real matrices, illustrate the theoretical results.
With the explosive growth in the demand of spectrum, conventional radio frequency systems are increasingly facing the challenge of catering to high-speed transmissions. Visible light communication (VLC) has been consi...
详细信息
With the explosive growth in the demand of spectrum, conventional radio frequency systems are increasingly facing the challenge of catering to high-speed transmissions. Visible light communication (VLC) has been considered as a promising supplement technique, since it can provide efficient energy efficiency and unlimited bandwidth. However, the nonlinearity effect is one of the fundamental problems in the VLC systems, which usually brings distortion to the transmitted signal and deteriorates the system performance. Aiming at this problem, numerous post-distortion schemes are proposed, where the kernel methods over reproducing kernel Hilbert spaces are identified to have successful applications. In this paper, we present a new post-distortion method with low computational cost, by utilizing the conjugategradient (CG) based kernel affine projection algorithm (KAPA), where the objective parameters are updated adaptively along the conjugate direction. Simulation results show that the proposed CG-KAPA based post-distorter can efficiently mitigate nonlinear impairment in VLC systems, which can provide better bit error rate performance with fast convergence over the commonly-used gradient descent algorithms.
Minimizing two different upper bounds of the matrix which generates search directions of the nonlinear conjugategradient method proposed by Dai and Liao, two modified conjugategradient methods are proposed. Under pr...
详细信息
Minimizing two different upper bounds of the matrix which generates search directions of the nonlinear conjugategradient method proposed by Dai and Liao, two modified conjugategradient methods are proposed. Under proper conditions, it is briefly shown that the methods are globally convergent when the line search fulfills the strong Wolfe conditions. Numerical comparisons between the implementations of the proposed methods and the conjugategradient methods proposed by Hager and Zhang, and Dai and Kou, are made on a set of unconstrained optimization test problems of the CUTEr collection. The results show the efficiency of the proposed methods in the sense of the performance profile introduced by Dolan and More. (C) 2013 Elsevier B.V. All rights reserved.
Satisfying in the sufficient descent condition is a strength of a conjugategradient method. Here, it is shown that under the Wolfe line search conditions the search directions generated by the memoryless BFGS conjuga...
详细信息
Satisfying in the sufficient descent condition is a strength of a conjugategradient method. Here, it is shown that under the Wolfe line search conditions the search directions generated by the memoryless BFGS conjugate gradient algorithm proposed by Shanno satisfy the sufficient descent condition for uniformly convex functions.
In this paper, we propose a new class of conjugate gradient algorithms for training neural networks which is based on a new modified nonmonotone scheme proposed by Shi and Wang (2011). The utilization of a nonmonotone...
详细信息
In this paper, we propose a new class of conjugate gradient algorithms for training neural networks which is based on a new modified nonmonotone scheme proposed by Shi and Wang (2011). The utilization of a nonmonotone strategy enables the training algorithm to overcome the case where the sequence of iterates runs into the bottom of a curved narrow valley, a common occurrence in neural network training process. Our proposed class of methods ensures sufficient descent, avoiding thereby the usual inefficient restarts and it is globally convergent under mild conditions. Our experimental results provide evidence that the proposed nonmonotone conjugategradient training methods are efficient, outperforming classical methods, proving more stable, efficient and reliable learning. (C) 2015 Elsevier Inc. All rights reserved.
Most geometric computer vision problems involve orthogonality constraints. An important subclass of these problems is subspace estimation, which can be equivalently formulated into an optimization problem on Grassmann...
详细信息
Most geometric computer vision problems involve orthogonality constraints. An important subclass of these problems is subspace estimation, which can be equivalently formulated into an optimization problem on Grassmann manifolds. In this paper, we propose to use the conjugate gradient algorithm on Grassmann manifolds for robust subspace estimation in conjunction with the recently introduced generalized projection based M-Estimator (gpbM). The gpbM method is an elemental subset-based robust estimation algorithm that can process heteroscedastic data without any user intervention. We show that by optimizing the orthogonal parameter matrix on Grassmann manifolds, the performance of the gpbM algorithm improves significantly. Results on synthetic and real data are presented. (c) 2011 Elsevier B.V. All rights reserved.
At the beginning of this century, which is characterized by huge flows of emerging data, Dai and Liao proposed a pervasive conjugacy condition that triggered the interest of many optimization scholars. Recognized as a...
详细信息
At the beginning of this century, which is characterized by huge flows of emerging data, Dai and Liao proposed a pervasive conjugacy condition that triggered the interest of many optimization scholars. Recognized as a sophisticated conjugategradient (CG) algorithm after about two decades, here we share our visions and thoughts on the method in the framework of a review study. In this regard, we first discuss the modified Dai-Liao methods based on the modified secant equations given in the literature, mostly with the aim of applying the objective function values in addition to the gradient information. Then, several adaptive, sort of optimal choices for the parameter of the method are studied. Especially, we devote a part of our study to the modified versions of the Hager-Zhang and Dai-Kou CG algorithms, being well-known members of the Dai-Liao class of CG methods. Extensions of the classical CG methods based on the Dai-Liao approach are also reviewed. Finally, we discuss the optimization models of practical disciplines that have been addressed by the Dai-Liao approach, including the nonlinear systems of equations, image restoration and compressed sensing.
It is pointed out that the so called momentum method, much used in the neural network literature as an acceleration of the backpropagation method, is a stationary version of the conjugategradient method. Connections ...
详细信息
It is pointed out that the so called momentum method, much used in the neural network literature as an acceleration of the backpropagation method, is a stationary version of the conjugategradient method. Connections with the continuous optimization method known as heavy ball with friction are also made. In both cases, adaptive (dynamic) choices of the so called learning rate and momentum parameters are obtained using a control Liapunov function analysis of the system. (C) 2003 Elsevier Ltd. All rights reserved.
暂无评论