The stochastic approximation algorithm (SAA), starting from the pioneer work by Robbins and Monro in 1950s, has been successfully applied in systems and control, statistics, machine learning, and so forth. In this pap...
详细信息
The stochastic approximation algorithm (SAA), starting from the pioneer work by Robbins and Monro in 1950s, has been successfully applied in systems and control, statistics, machine learning, and so forth. In this paper, we will review the development of SAA in China, to be specific, the stochastic approximation algorithm with expanding truncations (SAAWET) developed by Han-Fu Chen and his colleagues during the past 35 years. We first review the historical development for the centralized algorithm including the probabilistic method (PM) and the ordinary differential equation (ODE) method for SAA and the trajectory-subsequence method for SAAWET. Then, we will give an application example of SAAWET to the recursive principal component analysis. We will also introduce the recent progress on SAAWET in a networked and distributed setting, named the distributed SAAWET (DSAAWET).
In this paper, a class of stochastic extended vertical linear complementarity problems is studied as an extension of the stochastic linear complementarity problem. The expected residual minimization (ERM) formulation ...
详细信息
In this paper, a class of stochastic extended vertical linear complementarity problems is studied as an extension of the stochastic linear complementarity problem. The expected residual minimization (ERM) formulation of this stochastic extended vertical complementarity problem is proposed based on an NCP function. We study the corresponding properties of the ERM problem, such as existence of solutions, coercive property and differentiability. Finally, we propose a descent stochasticapproximation method for solving this problem. A comprehensive convergence analysis is given. A number of test examples are constructed and the numerical results are presented.
Important information concerning a multivariate data set, such as modal regions, is contained in the derivatives of the probability density or regression functions. Despite this importance, nonparametric estimation of...
详细信息
Important information concerning a multivariate data set, such as modal regions, is contained in the derivatives of the probability density or regression functions. Despite this importance, nonparametric estimation of higher order derivatives of the density or regression functions have received only relatively scant attention. The main purpose of the present work is to investigate general recursive kernel type estimators of function derivatives. We establish the central limit theorem for the proposed estimators. We discuss the optimal choice of the bandwidth by using the plug in methods. We obtain also the pointwise MDP of these estimators. Finally, we investigate the performance of the methodology for small samples through a short simulation study.
In the paper, recursive identification of the nonlinear nonparametric system is considered, where measurements for identification are subject to an event-triggered scheme and the system itself is of a finite impulse r...
详细信息
In the paper, recursive identification of the nonlinear nonparametric system is considered, where measurements for identification are subject to an event-triggered scheme and the system itself is of a finite impulse response (FIR) type. First, an adaptive event detector is designed and the measurements for identification are transmitted depending whether the value of the detector is 1 or 0. Second, a recursive identification algorithm is proposed with the help of a kernel-based stochastic approximation algorithm with expanding truncations (SAAWET). Third, under some reasonable assumptions, the estimates are proved to be strongly consistent, i.e., converging almost surely to the values of the unknown function in the system at any fixed points, and the transmission rate of the detector is analyzed as well. Finally, numerical examples are given to justify the theoretical results.
In this research paper, we apply the stochasticapproximation method to define a class of recursive kernel estimator of the conditional extreme value index. We investigate the properties of the proposed recursive esti...
详细信息
In this research paper, we apply the stochasticapproximation method to define a class of recursive kernel estimator of the conditional extreme value index. We investigate the properties of the proposed recursive estimator and compare them to those pertaining to Hill's non-recursive kernel estimator. We show that using some optimal parameters, the proposed recursive estimator defined by the stochastic approximation algorithm proves to be very competitive to Hill's non-recursive kernel estimator. Finally, the theoretical results are confirmed through simulation experiments and illustrated using a real dataset about Malaria in Senegalese children.
The paper considers the adaptive regulation for the Hammerstein and Wiener systems with event-triggered *** authors adopt a direct approach,i.e.,without identifying the unknown parameters and functions within the syst...
详细信息
The paper considers the adaptive regulation for the Hammerstein and Wiener systems with event-triggered *** authors adopt a direct approach,i.e.,without identifying the unknown parameters and functions within the systems,adaptive regulators are directly designed based on the event-triggered observations on the regulation *** adaptive regulators belong to the stochastic approximation algorithms and under moderate assumptions,the authors prove that the adaptive regulators are optimal for both the Hammerstein and Wiener systems in the sense that the squared regulation errors are asymptotically *** authors also testify the theoretical results through simulation studies.
The ever-increasing needs of supporting real-time applications have spurred new studies on minimizing Age-of-Information (AoI), a novel metric characterizing the data freshness of the system. This work studies the sin...
详细信息
The ever-increasing needs of supporting real-time applications have spurred new studies on minimizing Age-of-Information (AoI), a novel metric characterizing the data freshness of the system. This work studies the single-queue information update system and strengthens the seminal results of Sun et al. on the following fronts: (i) When designing the optimal offline schemes with full knowledge of the delay distributions, a new fixed-point-based method is proposed with quadratic convergence rate, an order-of-magnitude improvement over the state-of-the-art;(ii) When the distributional knowledge is unavailable (which is the norm in practice), two new low-complexity online algorithms are proposed, which provably attain the optimal average AoI penalty;and (iii) the online schemes also admit a modular architecture, which allows the designer to upgrade certain components to handle additional practical challenges. Two such upgrades are proposed for the situations: (iii.1) The AoI penalty function is also unknown and must be estimated on the fly, and (iii.2) the unknown delay distribution is Markovian instead of i.i.d. The performance of our schemes is either provably optimal or within 3% of the omniscient optimal offline solutions in all simulation scenarios.
In this paper, we propose and investigate a new kernel regression estimators based on the two-time-scale stochastic approximation algorithm in the case of independent functional data. We study the properties of the pr...
详细信息
In this paper, we propose and investigate a new kernel regression estimators based on the two-time-scale stochastic approximation algorithm in the case of independent functional data. We study the properties of the proposed recursive estimators and compare them with the recursive estimators based on single-time-scale stochasticalgorithm proposed by Slaoui and to the non-recursive estimator proposed by Slaoui. It turns out that, with an adequate choice of the parameters, the proposed two-time-scale estimators perform better than the recursive estimators constructed using single-time-scale stochasticalgorithm. We corroborate these theoretical results through some simulations and two real datasets.
In the present paper, we are concerned with a generalized kernel estimators defined by the stochastic approximation algorithm in the case of dependent functional data. We establish the central limit theorem for the pr...
详细信息
In the present paper, we are concerned with a generalized kernel estimators defined by the stochastic approximation algorithm in the case of dependent functional data. We establish the central limit theorem for the proposed estimators under some mild conditions. We then approach the distribution of the bias distribution of our estimate by the bootstrapped distribution when it is conditioned by the data using the Kolmogorov distance.
In this paper we show how one can implement in practice the bandwidth selection in deconvolution recursive kernel estimators of a probability density function defined by the stochastic approximation algorithm. We cons...
详细信息
In this paper we show how one can implement in practice the bandwidth selection in deconvolution recursive kernel estimators of a probability density function defined by the stochastic approximation algorithm. We consider the so called super smooth case where the characteristic function of the known distribution decreases exponentially. We show that, using the proposed bandwidth selection and some special stepsizes, the proposed recursive estimator will be very competitive to the nonrecursive one in terms of estimation error and much better in terms of computational costs. We corroborate these theoretical results through simulations and a real dataset.
暂无评论