Reduced-order identificationalgorithms are usually used in machine learning and big data technologies, where the large-scale systems widely exist. For large-scale system identification, traditional least squares algo...
详细信息
Reduced-order identificationalgorithms are usually used in machine learning and big data technologies, where the large-scale systems widely exist. For large-scale system identification, traditional least squares algorithm involves high-order matrix inverse calculation, while traditional gradient descent algorithm has slow convergence rates. The reduced-order algorithm proposed in this paper has some advantages over the previous work: (1) via sequential partitioning of the parameter vector, the calculation of the inverse of a high-order matrix can be reduced to low-order matrix inverse calculations;(2) has a better conditioned information matrix than that of the gradient descent algorithm, thus has faster convergence rates;(3) its convergence rates can be increased by using the Aitken acceleration method, therefore the reduced-order based Aitken algorithm is at least quadratic convergent and has no limitation on the step-size. The properties of the reduced-order algorithm are also given. Simulation results demonstrate the effectiveness of the proposed algorithm. (c) 2024 Elsevier Ltd. All rights are reserved, including those for text and data mining, AI training, and similar technologies.
作者：
Chen, Guang-YongLin, XinXue, PengGan, MinFuzhou Univ
Coll Comp & Data Sci Fuzhou 350116 Peoples R China Minist Educ
Fujian Key Lab Network Comp & Intelligent Informat Key Lab Intelligent Metro Univ Fujian Fuzhou 350108 Peoples R China Minist Educ
Engn Res Ctr Big Data Intelligence Fuzhou 350108 Peoples R China Qingdao Univ
Coll Comp Sci & Technol Qingdao 266071 Peoples R China
Separable nonlinear models are pervasively employed in diverse disciplines, such as system identification, signal analysis, electrical engineering, and machine learning. Identifying these models inherently poses a non...
详细信息
Separable nonlinear models are pervasively employed in diverse disciplines, such as system identification, signal analysis, electrical engineering, and machine learning. Identifying these models inherently poses a non-convex optimization challenge. While gradient descent (GD) is a commonly adopted method, it is often plagued by suboptimal convergence rates and is highly dependent on the appropriate choice of step size. To mitigate these issues, we introduce an augmented GD algorithm enhanced with Anderson acceleration (AA), and propose a hierarchical GD with Anderson acceleration (H-AAGD) method for efficient identification of separable nonlinear models. This novel approach transcends the conventional step size constraints of GD algorithms and considers the coupling relationships between different parameters during the optimization process, thereby enhancing the efficiency of the solution-finding process. Unlike the Newton method, our algorithm obviates the need for computing the inverse of the Hessian matrix, simplifying the computational process. Additionally, we theoretically analyze the convergence and complexity of the algorithm and validate its effectiveness through a series of numerical experiments.
In this study, the authors consider the parameter estimation problem of the response signal from a highly non-linear dynamical system. The step response experiment is taken for generating the measured data. Considerin...
详细信息
In this study, the authors consider the parameter estimation problem of the response signal from a highly non-linear dynamical system. The step response experiment is taken for generating the measured data. Considering the stochastic disturbance in the industrial process and using the gradient search, a multi-innovation stochastic gradient algorithm is proposed through expanding the scalar innovation into an innovation vector in order to obtain more accurate parameter estimates. Furthermore, a hierarchical identification algorithm is derived by means of the decomposition technique and interaction estimation theory. Regarding to the coupled parameter problem between subsystems, the authors put forward the scheme of replacing the unknown parameters with their previous parameter estimates to realise the parameter estimation algorithm. Finally, several examples are provided to access and compare the behaviour of the proposed identification techniques.
Aitken gradient descent (AGD) algorithm takes some advantages over the standard gradient descent and Newton methods: 1) can achieve at least quadratic convergence in general;2) does not require the Hessian matrix inve...
详细信息
Aitken gradient descent (AGD) algorithm takes some advantages over the standard gradient descent and Newton methods: 1) can achieve at least quadratic convergence in general;2) does not require the Hessian matrix inversion;3) has less computational efforts. When using the AGD method for a considered model, the iterative function should be unchanging during all the iterations. This article proposes a hierarchical AGD algorithm for separable nonlinear models based on stage greedy method. The linear parameters are estimated using the least squares algorithm, and the nonlinear parameters are updated based on the AGD algorithm. Since the iterative function is changing at each iteration, a stage AGD algorithm is introduced. The convergence properties and simulation examples show effectiveness of the proposed algorithm.
For the dual-rate system, such as the process of space teleoperation whose control signals is partly determined by delayed feedback states, the state values and system parameters are coupled and influenced each other,...
详细信息
For the dual-rate system, such as the process of space teleoperation whose control signals is partly determined by delayed feedback states, the state values and system parameters are coupled and influenced each other, which are hard to be estimated simultaneously. In this paper, we propose a novel method for this problem. Firstly, considering the asynchronism of the input and output sampling signals, an auxiliary model is modeled as a medium to the state and output functions. Secondly, the Kalman prediction algorithm is improved to estimate the state values at output signals of the dual-rate system. The general step is using the output estimated errors in original and auxiliary systems to modify the estimated state values of the auxiliary model, and then the unknown state values in original system is defined by the ones in auxiliary model. Based on improved Kalman algorithm and hierarchical identification algorithm, we present the detailed procedures of state estimation and parameter identification method for the dual-rate system. The processes of state estimation and parameter identification are calculated and modified alternately. Finally, the simulation results reveal that the state and parameters both approach to the real values and the state values converge faster than the parameters.
Many inverse problems in machine learning, system identification, and image processing include nuisance parameters, which are important for the recovering of other parameters. Separable nonlinear optimization problems...
详细信息
Many inverse problems in machine learning, system identification, and image processing include nuisance parameters, which are important for the recovering of other parameters. Separable nonlinear optimization problems fall into this category. The special separable structure in these problems has inspired several efficient optimization strategies. A well-known method is the variable projection (VP) that projects out a subset of the estimated parameters, resulting in a reduced problem that includes fewer parameters. The expectation maximization (EM) is another separated method that provides a powerful framework for the estimation of nuisance parameters. The relationships between EM and VP were ignored in previous studies, though they deal with a part of parameters in a similar way. In this article, we explore the internal relationships and differences between VP and EM. Unlike the algorithms that separate the parameters directly, the hierarchical identification algorithm decomposes a complex model into several linked submodels and identifies the corresponding parameters. Therefore, this article also studies the difference and connection between the hierarchicalalgorithm and the parameter-separated algorithms like VP and EM. In the numerical simulation part, Monte Carlo experiments are performed to further compare the performance of different algorithms. The results show that the VP algorithm usually converges faster than the other two algorithms and is more robust to the initial point of the parameters.
The viscoelastic relaxation spectrum is vital for constitutive models and for insight into the mechanical properties of materials, since, from the relaxation spectrum, other material functions used to describe rheolog...
详细信息
The viscoelastic relaxation spectrum is vital for constitutive models and for insight into the mechanical properties of materials, since, from the relaxation spectrum, other material functions used to describe rheological properties can be uniquely determined. The spectrum is not directly accessible via measurement and must be recovered from relaxation stress or oscillatory shear data. This paper deals with the problem of the recovery of the relaxation time spectrum of linear viscoelastic material from discrete-time noise-corrupted measurements of a relaxation modulus obtained in the stress relaxation test. A two-level identification scheme is proposed. In the lower level, the regularized least-square identification combined with generalized cross-validation is used to find the optimal model with an arbitrary time-scale factor. Next, in the upper level, the optimal time-scale factor is determined to provide the best fit of the relaxation modulus to experiment data. The relaxation time spectrum is approximated by a finite series of power-exponential basis functions. The related model of the relaxation modulus is proved to be given by compact analytical formulas as the products of power of time and the modified Bessel functions of the second kind. The proposed approach merges the technique of an expansion of a function into a series of independent basis functions with the least-squares regularized identification and the optimal choice of the time-scale factor. Optimality conditions, approximation error, convergence, noise robustness and model smoothness are studied analytically. Applicability ranges are numerically examined. These studies have proved that using a developed model and algorithm, it is possible to determine the relaxation spectrum model for a wide class of viscoelastic materials. The model is smoothed and noise robust;small model errors are obtained for the optimal time-scale factors. The complete scheme of the hierarchical computations is outlined, whic
暂无评论