Quantitative Error Bounds for Scaling Limits of Stochastic Iterative Algorithms

作     者：Wang, Xiaoyu Kasprzak, Mikolaj J. Negrea, Jeffrey Bourguin, Solesne Huggins, Jonathan H. 

作者机构：Department of Mathematics & Statistics Boston University United States Department of Information Systems Data Analytics & Operations ESSEC Business School France Department of Statistics & Actuarial Science University of Waterloo Canada Faculty of Computing & Data Sciences Boston University United States 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2025年

核心收录：

主　　题：Continuous time systems 

摘      要：Stochastic iterative algorithms, including stochastic gradient descent (SGD) and stochastic gradient Langevin dynamics (SGLD), are widely utilized for optimization and sampling in large-scale and high-dimensional problems in machine learning, statistics, and engineering. Numerous works have bounded the parameter error in, and characterized the uncertainty of, these approximations. One common approach has been to use scaling limit analyses to relate the distribution of algorithm sample paths to a continuous-time stochastic process approximation, particularly in asymptotic setups. Focusing on the univariate setting, in this paper, we build on previous work to derive non-asymptotic functional approximation error bounds between the algorithm sample paths and the Ornstein-Uhlenbeck approximation using an infinite-dimensional version of Stein s method of exchangeable pairs. We show that this bound implies weak convergence under modest additional assumptions and leads to a bound on the error of the variance of the iterate averages of the algorithm. Furthermore, we use our main result to construct error bounds in terms of two common metrics: the Lévy-Prokhorov and bounded Wasserstein distances. Our results provide a foundation for developing similar error bounds for the multivariate setting and for more sophisticated stochastic approximation *** Codes 60F17 (Primary) 60J60, 62-08, 68T05, 62E17 (Secondary) Copyright © 2025, The Authors. All rights reserved.