This paper proposes a joint variable-based gradient descent algorithm (Joint-GD) and a variable projection (VP)-based gradient descent algorithm (VP-GD) for separable nonlinear models. The VP algorithm takes advantage...
详细信息
This paper proposes a joint variable-based gradient descent algorithm (Joint-GD) and a variable projection (VP)-based gradient descent algorithm (VP-GD) for separable nonlinear models. The VP algorithm takes advantage of the separability property of variables to reduce the dimensionality of the parameters, which makes the convergence rates faster. In order to speed up the convergence of the gradient descent algorithm, the aitken acceleration technique is introduced in the algorithms, which is second-order convergent. Moreover, the aitken-based methods are robust to the step-size, therefore they can be widely used in engineering practices. The numerical simulation shows the effectiveness of the proposed algorithms.
A robust standard gradient descent (SGD) algorithm for ARX models using the aitkenacceleration method is developed. Considering that the SGD algorithm has slow convergence rates and is sensitive to the step size, a r...
详细信息
A robust standard gradient descent (SGD) algorithm for ARX models using the aitkenacceleration method is developed. Considering that the SGD algorithm has slow convergence rates and is sensitive to the step size, a robust and accelerative SGD (RA-SGD) algorithm is derived. This algorithm is based on the aitkenacceleration method, and its convergence rate is improved from linear convergence to at least quadratic convergence in general. Furthermore, the RA-SGD algorithm is always convergent with no limitation of the step size. Both the convergence analysis and the simulation examples demonstrate that the presented algorithm is effective.
aitken gradient descent (AGD) algorithm takes some advantages over the standard gradient descent and Newton methods: 1) can achieve at least quadratic convergence in general;2) does not require the Hessian matrix inve...
详细信息
aitken gradient descent (AGD) algorithm takes some advantages over the standard gradient descent and Newton methods: 1) can achieve at least quadratic convergence in general;2) does not require the Hessian matrix inversion;3) has less computational efforts. When using the AGD method for a considered model, the iterative function should be unchanging during all the iterations. This article proposes a hierarchical AGD algorithm for separable nonlinear models based on stage greedy method. The linear parameters are estimated using the least squares algorithm, and the nonlinear parameters are updated based on the AGD algorithm. Since the iterative function is changing at each iteration, a stage AGD algorithm is introduced. The convergence properties and simulation examples show effectiveness of the proposed algorithm.
Systems with unknown structures widely exist in engineering practices. In this paper, a Volterra series is applied to approximate the dynamics of systems. Due to the special structure of the Volterra model, the approx...
详细信息
Systems with unknown structures widely exist in engineering practices. In this paper, a Volterra series is applied to approximate the dynamics of systems. Due to the special structure of the Volterra model, the approximated model has a high order and some redundant terms. A regularised term is introduced to pick out these redundant terms, and then a proximal gradient method is provided to estimate the unknown parameters of the Volterra model. Furthermore, an accelerated technique is proposed to increase the convergence rates. The advantages of this algorithm are as follows: (1) can pick out the redundant terms without any prior knowledge of the model;(2) has fast convergence rates;and (3) is robust to the step-size. The effectiveness of the proposed algorithm is further substantiated through a simulation example.
The 2D integral equations are computed by an integrated iteration algorithm of modified successive approximation method (MSAM) and aitken acceleration technique. The longitudinally stratified virgin formation is taken...
详细信息
The 2D integral equations are computed by an integrated iteration algorithm of modified successive approximation method (MSAM) and aitken acceleration technique. The longitudinally stratified virgin formation is taken as the background and the computational region is restricted within the borehole and invasion zone in the algorithm, thus the algorithm is qualified for the virtues of small number of unknowns, fast converging speed and high accuracy. The response of the electromagnetic wave resistivity MWD tool in cylindrically symmetrical 2D formation is simulated by the algorithm. The results have shown that the amplitude attenuation and phase shift are affected differently by the borehole, invasion and surrounding shale, and that their radial depth of investigation and vertical resolution are also different. The bed boundary can be accurately located by the crossover point of the compensated amplitude attenuation and phase shift resistivities.
暂无评论