检索结果-内蒙古大学图书馆

Two-Armed Bandit Problem and Batch Version of the mirror descent algorithm

AUTOMATION AND REMOTE CONTROL 2022年第8期83卷 1288-1307页

作者： Kolnogorov, A., V Nazin, A., V Shiyan, D. N. Yaroslav The Wise Novgorod State Univ Novgorod 173003 Russia Russian Acad Sci Trapeznikov Inst Control Sci Moscow 117997 Russia

We consider the minimax setup for the two-armed bandit problem as applied to data processing if there are two alternative processing methods with different a priori unknown efficiencies. One should determine the most efficient method and provide its predominant application. To this end, we use the mirror descent algorithm (MDA). It is well known that the corresponding minimax risk has the order of N-1/2, where N is the amount of processed data, and this bound is order sharp. We propose a batch version of the MDA which allows processing data by packets;this is especially important if parallel data processing can be provided. In this case, the processing time is determined by the number of batches rather than the total amount of data. Unexpectedly, it has turned out that the batch version behaves unlike the ordinary one even if the number of packets is large. Moreover, the batch version provides a considerably lower minimax risk;i.e., it substantially improves the control performance. We explain this result by considering another batch modification of the MDA whose behavior is close to the behavior of the ordinary version and the minimax risk is close as well. Our estimates use invariant descriptions of the algorithms based on Gaussian approximations of income in the batches of data in the domain of "close" distributions and are obtained by Monte-Carlo simulation.

关键词： two-armed bandit problem minimax approach mirror descent algorithm EXP3 batch processing

来源：评论

学校读者我要写书评

暂无评论

Robust Analysis of Almost Sure Convergence of Zeroth-Order mirror descent algorithm

引用

IEEE CONTROL SYSTEMS LETTERS 2023年 7卷 1933-1938页

作者： Paul, Anik Kumar Mahindrakar, Arun D. Kalaimani, Rachel K. Indian Inst Technol Madras Dept Elect Engn Chennai 600036 India

This letter presents an almost sure convergence of the zeroth-order mirror descent (ZOMD) algorithm. The algorithm admits non-smooth convex functions and a biased oracle which only provides noisy function value at any desired point. We approximate the subgradient of the objective function using Nesterov's Gaussian Approximation (NGA) with certain alternations suggested by some practical applications. We prove an almost sure convergence of the iterates' function value to the neighbourhood of optimal function value, which can not be made arbitrarily small, a manifestation of a biased oracle. This letter ends with a concentration inequality, which is a finite time analysis that predicts the likelihood that the function value of the iterates is in the neighbourhood of the optimal value at any finite iteration.

关键词： Almost sure convergence subgradient approximation mirror descent algorithm

来源：评论

学校读者我要写书评

暂无评论

One-Armed Bandit Problem and the mirror descent algorithm

引用

DOKLADY MATHEMATICS 2024年第SUPPL2期110卷 S399-S408页

作者： Shiyan, D. N. Yaroslav The Wise Novgorod State Univ Veliky Novgorod 173003 Novgorod Oblast Russia

The application of the mirror descent algorithm (MDA) in the one-armed bandit problem in the minimax setting in relation to data processing has been considered. This problem has also been known as a game with nature, in which the payoff function of the player is the mathematical expectation of the total income. The player must determine the most effective method of the two available ones during the control process and ensure its preferential use. In this case, the a priori efficiency of one of the methods is known. In this paper, a modification of the MDA that makes it possible to improve the control efficiency by using additional information has been considered. The proposed strategy preserves the characteristic property of strategies for one-armed bandits: if a known action is applied once, it will be applied until the end of control. Modifications for the algorithm for single processing and for its batch version have been considered. Batch processing is interesting in that the total processing time is determined by the number of packets, and not by the original amount of data, with the possibility of providing parallel processing of data in packets. For the proposed algorithms, the optimal values of the adjustable parameters have been calculated using Monte Carlo simulation and minimax risk estimates have been obtained.

关键词： two-armed bandit problem one-armed bandit problem minimax approach mirror descent algorithm EXP3 batch processing

来源：评论

学校读者我要写书评

暂无评论

Almost Sure Convergence and Non-Asymptotic Concentration Bounds for Stochastic mirror descent algorithm

引用

IEEE CONTROL SYSTEMS LETTERS 2024年 8卷 2397-2402页

作者： Paul, Anik Kumar Mahindrakar, Arun D. Kalaimani, Rachel K. Indian Inst Technol Madras Dept Elect Engn Chennai 600036 India

This letter investigates the convergence and concentration properties of the Stochastic mirror descent (SMD) algorithm utilizing biased stochastic subgradients. We establish the almost sure convergence of the algorithm's iterates under the assumption of diminishing bias. Furthermore, we derive concentration bounds for the discrepancy between the iterates' function values and the optimal value, based on standard assumptions. Subsequently, leveraging the assumption of Sub-Gaussian noise in stochastic subgradients, we present refined concentration bounds for this discrepancy.

关键词： mirror descent algorithm almost sure convergence concentration inequality sub-gaussian random vectors mirror descent algorithm almost sure convergence concentration inequality sub-gaussian random vectors

来源：评论

学校读者我要写书评

暂无评论

Parallel Version of the mirror descent algorithm for the Two-Armed Bandit Problem 3

Parallel Version of the Mirror Descent Algorithm for the Two...

引用

3rd International Conference on Mathematics and Computers in Sciences and in Industry (MCSI)

作者： Kolnogorov, Alexander Shiyan, Dmitry Yaroslav The Wise Novgorod State Univ B St Petersburgskaya Str 41 Velikiy Novgorod 173003 Russia

We propose some modified versions of the mirror descent algorithm for the two-armed bandit problem which allow parallel processing of data. Using Monte-Carlo simulations, we estimate the minimax risk for this versions.

ISBN: (纸本)9781509009732

关键词： two-armed bandit problem control in a random environment minimax and bayesian approaches robust control mirror descent algorithm parallel processing

来源：评论

学校读者我要写书评

暂无评论

Reinforcement learning with constraint based on mirror descent algorithm

引用

RESULTS IN CONTROL AND OPTIMIZATION 2021年 4卷

作者： Miyashita, Megumi Kondo, Toshiyuki Yano, Shiro Tokyo Univ Agr & Technol Grad Sch Engn Dept Elect & Informat Engn 2-24-16 Naka Cho Koganei Tokyo Japan Tokyo Univ Agr & Technol Inst Engn Div Adv Informat Technol & Comp Sci 2-24-16 Naka Cho Koganei Tokyo Japan

An important issue in reinforcement learning is to make the agent avoid the dangers and risks during the task such as physical collisions. We propose the reinforcement learning algorithm based on the Comirror algorithm, named CoMDS, for the problem that has a functional constraint. Besides, we modify the proposed algorithm CoMDS to Gaussian CoMDS for practical use. We evaluate our algorithms with the via -point task of a planar robotic arm with a forbidden area, that employs as a constraint, in the simulator. As a result, we find that Gaussian CoMDS explores the policy while satisfying the constraint.

关键词： Constrained optimization mirror descent algorithm

来源：评论

学校读者我要写书评

暂无评论

ERGODIC mirror descent

引用

SIAM JOURNAL ON OPTIMIZATION 2012年第4期22卷 1549-1578页

作者： Duchi, John C. Agarwal, Alekh Johansson, Mikael Jordan, Michael I. Univ Calif Berkeley Dept Elect Engn & Comp Sci Berkeley CA 94720 USA Microsoft Res New York NY USA Royal Inst Technol KTH Sch Elect Engn Stockholm Sweden

We generalize stochastic subgradient descent methods to situations in which we do not receive independent samples from the distribution over which we optimize, instead receiving samples coupled over time. We show that as long as the source of randomness is suitably ergodic it converges quickly enough to a stationary distribution-the method enjoys strong convergence guarantees, both in expectation and with high probability. This result has implications for stochastic optimization in high-dimensional spaces, peer-to-peer distributed optimization schemes, decision problems with dependent data, and stochastic optimization problems over combinatorial spaces.

关键词： convex programming stochastic optimization Markov chain Monte Carlo sampling mixing mirror descent algorithm

来源：评论

学校读者我要写书评

暂无评论

Validation analysis of mirror descent stochastic approximation method

引用

MATHEMATICAL PROGRAMMING 2012年第2期134卷 425-458页

作者： Lan, Guanghui Nemirovski, Arkadi Shapiro, Alexander Univ Florida Gainesville FL 32611 USA Georgia Inst Technol Atlanta GA 30332 USA

The main goal of this paper is to develop accuracy estimates for stochastic programming problems by employing stochastic approximation (SA) type algorithms. To this end we show that while running a mirror descent Stochastic Approximation procedure one can compute, with a small additional effort, lower and upper statistical bounds for the optimal objective value. We demonstrate that for a certain class of convex stochastic programs these bounds are comparable in quality with similar bounds computed by the sample average approximation method, while their computational cost is considerably smaller.

关键词： Stochastic approximation Sample average approximation method Stochastic programming Monte Carlo sampling mirror descent algorithm Prox-mapping Optimality bounds Large deviations estimates Asset allocation problem Conditional value-at-risk

来源：评论

学校读者我要写书评

暂无评论

mirror descent Strikes Again: Optimal Stochastic Convex Optimization under Infinite Noise Variance 35

Mirror Descent Strikes Again: Optimal Stochastic Convex Opti...

引用

35th Conference on Learning Theory (COLT)

作者： Vural, Nuri Mert Yu, Lu Balasubramanian, Krishnakumar Volgushev, Stanislav Erdogdu, Murat A. Univ Toronto Dept Comp Sci Toronto ON Canada Vector Inst Toronto ON Canada Univ Toronto Dept Stat Sci Toronto ON Canada Univ Calif Davis Dept Stat Davis CA 95616 USA

We study stochastic convex optimization under infinite noise variance. Specifically, when the stochastic gradient is unbiased and has uniformly bounded (1+ k)-th moment, for some k is an element of (0;1], we quantify the convergence rate of the Stochastic mirror descent algorithm with a particular class of uniformly convex mirror maps, in terms of the number of iterations, dimensionality and related geometric parameters of the optimization problem. Interestingly this algorithm does not require any explicit gradient clipping or normalization, which have been extensively used in several recent empirical and theoretical works. We complement our convergence results with information-theoretic lower bounds showing that no other algorithm using only stochastic first-order oracles can achieve improved rates. Our results have several interesting consequences for devising online/streaming stochastic approximation algorithms for problems arising in robust statistics and machine learning.

关键词： mirror descent algorithm uniformly convex functions heavy-tailed gradient noise oracle complexity information-theoretic lower bounds

来源：评论

学校读者我要写书评

暂无评论

Quantizer-based distributed mirror descent for multi-agent convex optimization 33

Quantizer-based distributed mirror descent for multi-agent c...

引用

33rd Chinese Control and Decision Conference (CCDC)

作者： Xiong, Menghui Zhang, Baoyong Yuan, Deming Nanjing Univ Sci & Technol Sch Automat Nanjing 210094 Jiangsu Peoples R China

ISBN: (纸本)9781665440899

This paper is concerned with the constrained distributed multi-agent convex optimization problem over a time-varying network. We assume that the bit rate of the considered communication is limited, such that a uniform quantizer is applied in the process of exchanging information over the multi-agent network. Then a quantizer-based distributed mirror descent (QDMD) algorithm, which utilizes the Bregman divergence as the distance-measuring function, is developed for such optimization problem. The convergence result of the developed algorithm is also analyzed. By choosing the iteration step-size eta(t) = lambda/root t and quantization interval upsilon(t) = lambda/t with a prescribed parameter lambda, it is shown that the convergence rate of the QDMD algorithm can achieve O(1/root T), where T is the number of iterations.

关键词： mirror descent algorithm distributed optimization uniform quantizer multi-agent system

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：