Stochastic relaxed inertial forward-backward-forward splitting for monotone inclusions in Hilbert spaces

作     者：Cui, Shisheng Shanbhag, Uday V. Staudigl, Mathias Vuong, Phan Tu 

作者机构：Department of Industrial and Manufacturing Engineering Pennsylvania State University University ParkPA16802 United States Department of Data Science and Knowledge Engineering Maastricht University P.O. Box 616 MaastrichtNL-6200 MD Netherlands Mathematical Sciences University of Southampton Highfield SouthamptonSO17 1BJ United Kingdom 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2021年

核心收录：

主　　题：Stochastic systems 

摘      要：We consider monotone inclusions defined on a Hilbert space where the operator is given by the sum of a maximal monotone operator T and a single-valued monotone, Lipschitz continuous, and expectation-valued operator V. We draw motivation from the seminal work by Attouch and Cabot [1, 2] on relaxed inertial methods for monotone inclusions and present a stochastic extension of the relaxed inertial forward-backward-forward (RISFBF) method. Facilitated by an online variance reduction strategy via a mini-batch approach, we show that (RISFBF) produces a sequence that weakly converges to the solution set. Moreover, it is possible to estimate the rate at which the discrete velocity of the stochastic process vanishes. Under strong monotonicity, we demonstrate strong convergence, and give a detailed assessment of the iteration and oracle complexity of the scheme. When the mini-batch is raised at a geometric (polynomial) rate, the rate statement can be strengthened to a linear (suitable polynomial) rate while the oracle complexity of computing an ∊-solution improves to O(1/∊). Importantly, the latter claim allows for possibly biased oracles, a key theoretical advancement allowing for far broader applicability. By defining a restricted gap function based on the Fitzpatrick function, we prove that the expected gap of an averaged sequence diminishes at a sublinear rate of O(1/k) while the oracle complexity of computing a suitably defined ∊-solution is O(1/∊1+a) where a 1. Numerical results on two-stage games and an overlapping group Lasso problem illustrate the advantages of our method compared to stochastic forward-backward-forward (SFBF) and SA *** Codes 37H10, 49M37, 65C40, 65K15. © 2021, CC BY.