An usual problem in statistics consists in estimating the minimizer of a convex function. When we have to deal with large samples taking values in high dimensional spaces, stochastic gradient algorithms and their aver...
详细信息
An usual problem in statistics consists in estimating the minimizer of a convex function. When we have to deal with large samples taking values in high dimensional spaces, stochastic gradient algorithms and their averaged versions are efficient candidates. Indeed, (1) they do not need too much computational efforts, (2) they do not need to store all the data, which is crucial when we deal with big data, (3) they allow to simply update the estimates, which is important when data arrive sequentially. The aim of this work is to give asymptotic and non asymptotic rates of convergence of stochasticgradient estimates as well as of their averaged versions when the function we would like to minimize is only locally strongly convex.
Time-delay dynamic systems are widely existed in industrial applications owing to the measure, control or other processes. To carry out system analysis, control and fault diagnosis, the identification of time-delay sy...
详细信息
Time-delay dynamic systems are widely existed in industrial applications owing to the measure, control or other processes. To carry out system analysis, control and fault diagnosis, the identification of time-delay systems becomes more and more important. This article considers the identification of time-delay ARX models based on a novel two-stage algorithm. Firstly, a 2-copula criterion based time-delay estimation method is presented by using a measure of dependence between the model input and output. This method can obtain the time delay without the estimates of the parameters. Secondly, a multi-gradientalgorithm with adaptive stacking length is studied. This algorithm accelerates traditional stochastic gradient algorithm by taking several recent gradients in each iteration. The stacking length, that is, the number of the gradient used in a step, is determined by the Armijo criterion. The proposed algorithm is validated by numerical experiments and the modeling of unmanned aerial vehicle data.
stochastic gradient algorithms are more and more studied since they can deal efficiently and online with large samples in high dimensional spaces. In this paper, we first establish a Central Limit Theorem for these es...
详细信息
stochastic gradient algorithms are more and more studied since they can deal efficiently and online with large samples in high dimensional spaces. In this paper, we first establish a Central Limit Theorem for these estimates as well as for their averaged version in general Hilbert spaces. Moreover, since having the asymptotic normality of estimates is often unusable without an estimation of the asymptotic variance, we introduce a new recursive algorithm for estimating this last one, and we establish its almost sure rate of convergence as well as its rate of convergence in quadratic mean. Finally, two examples consisting in estimating the parameters of the logistic regression and estimating geometric quantiles are given. (C) 2019 Elsevier B.V. All rights reserved.
Time delay dynamic systems are widely existed due to sensors, actuators or other reasons. In this paper, a time delay FIR system is considered to model linear dynamic systems. The reason why the FIR model is selected ...
详细信息
Time delay dynamic systems are widely existed due to sensors, actuators or other reasons. In this paper, a time delay FIR system is considered to model linear dynamic systems. The reason why the FIR model is selected is to highlight the proposed time-delay estimation method and parameter identification algorithm, and avoid the impact of a complex model on readers' understanding of the proposed technologies. Firstly, to obtain an estimate of the time delay, a dependence measure based method is proposed. Unlike the optimization method that requires the parameter estimate and needs to round the estimated delay, the delay estimation method based on the 2-copula dependence measure can give accurate delay estimates independently of the parameters and without rounding. Secondly, to estimate the parameters, a variable stacking length multi-gradient identification algorithm is studied. The multi-gradient technique takes recent several gradients to accelerate the stochastic gradient algorithm. The stacking length, i.e., the number of gradients used in each iteration, is determined by the Wolfe-Powell criterion. The effectiveness is tested by numerical simulations and case study.
This paper addresses the combined estimation issues of the parameters and states for fractional-order Hammerstein state space systems with colored noises. An extended state estimator is derived by using the parameter ...
详细信息
This paper addresses the combined estimation issues of the parameters and states for fractional-order Hammerstein state space systems with colored noises. An extended state estimator is derived by using the parameter estimates to replace the unknown system parameters in Kalman filter. The hierarchical identification principle is introduced to solve the unknown parameters of measurement noises. By introducing the forgetting factor, an extended Kalman filtering-based hierarchical forgetting factor stochastic gradient algorithm is presented to estimate the unknown states, parameters and fractional-order. A numerical example is respectively presented to demonstrate the feasibility of the proposed identification algorithm. It can be seen that the estimation errors are relatively small, which reflects the proposed algorithms have good identification effect.
This paper considers the identification problem of linear regression model with quantized observations. We propose a modified stochastic gradient algorithm to estimate unknown parameters using the quanized observation...
详细信息
ISBN:
(纸本)9798350366907;9789887581581
This paper considers the identification problem of linear regression model with quantized observations. We propose a modified stochastic gradient algorithm to estimate unknown parameters using the quanized observations. Under the "weakest" excitation condition, we show that the proposed algorithm converges to the true parameters without requiring the commonly used i.i.d. conditions. Based on this, the convergence rate is further considered. Finally, simulation results are given to demonstrate the effectiveness of the proposed algorithm.
We propose a neural network training method based on the dynamic partition of the training data set. The main idea is to perform multiple iterative training on the neural network model. Each iteration of training spli...
详细信息
ISBN:
(纸本)9798350390780;9798350379228
We propose a neural network training method based on the dynamic partition of the training data set. The main idea is to perform multiple iterative training on the neural network model. Each iteration of training splits the training data set, and allows the neural network model to train independently on several sub datasets obtained by the segmentation. According to the error performance on the entire training data set of the neural network model trained on the sub-dataset, the optimal neural network model with the smallest error is selected as the initial network for the next iteration *** show that this training method based on dynamic segmentation of the training data set has two advantages compared with the traditional training method. One is to improve the training efficiency, because the sub-datasets obtained by each segmentation are independent of each other. Data set segmentation can be used for parallel training to accelerate training. The other is that this progressive training model of training data sets from small to large, from less to more, conforms to the changing law of human understanding of things, and makes the model have better training accuracy and generalization ability.
Online averaged stochastic gradient algorithms are more and more studied since (i) they can deal quickly with large sample taking values in high-dimensional spaces, (ii) they enable to treat data sequentially, (iii) t...
详细信息
Online averaged stochastic gradient algorithms are more and more studied since (i) they can deal quickly with large sample taking values in high-dimensional spaces, (ii) they enable to treat data sequentially, (iii) they are known to be asymptotically efficient. In this paper, we focus on giving explicit bounds of the quadratic mean error of the estimates, and this, without supposing that the function we would like to minimize is strongly convex or admits a bounded gradient.
Models incorporating uncertain inputs, such as random forces or material parameters, have been of increasing interest in PDE-constrained optimization. In this paper, we focus on the efficient numerical minimization of...
详细信息
Models incorporating uncertain inputs, such as random forces or material parameters, have been of increasing interest in PDE-constrained optimization. In this paper, we focus on the efficient numerical minimization of a convex and smooth tracking-type functional subject to a linear partial differential equation with random coefficients and box constraints. The approach we take is based on stochastic approximation where, in place of a true gradient, a stochasticgradient is chosen using one sample from a known probability distribution. Feasibility is maintained by performing a projection at each iteration. In the application of this method to PDE-constrained optimization under uncertainty, new challenges arise. We observe the discretization error made by approximating the stochasticgradient using finite elements. Analyzing the interplay between PDE discretization and stochastic error, we develop a mesh refinement strategy coupled with decreasing step sizes. Additionally, we develop a mesh refinement strategy for the modified algorithm using iterate averaging and larger step sizes. The effectiveness of the approach is demonstrated numerically for different random field choices.
In this article, we propose a new method for multiobjective optimization problems in which the objective functions are expressed as expectations of random functions. The present method is based on an extension of the ...
详细信息
In this article, we propose a new method for multiobjective optimization problems in which the objective functions are expressed as expectations of random functions. The present method is based on an extension of the classical stochastic gradient algorithm and a deterministic multiobjective algorithm, the Multiple gradient Descent algorithm (MGDA). In MGDA a descent direction common to all specified objective functions is identified through a result of convex geometry. The use of this common descent vector and the Pareto stationarity definition into the stochastic gradient algorithm makes the algorithm able to solve multiobjective problems. The mean square and almost sure convergence of this new algorithm are proven considering the classical stochastic gradient algorithm hypothesis. The algorithm efficiency is illustrated on a set of benchmarks with diverse complexity and assessed in comparison with two classical algorithms (NSGA-II, DMS) coupled with a Monte Carlo expectation estimator. (C) 2018 Elsevier B.V. All rights reserved.
暂无评论