版权所有:内蒙古大学图书馆 技术提供:维普资讯• 智图
内蒙古自治区呼和浩特市赛罕区大学西街235号 邮编: 010021
作者机构:Politecn Milan DEIB Milan Italy Facebook AI Res Paris France Ist Italiano Tecnol Genoa Italy
出 版 物:《JOURNAL OF MACHINE LEARNING RESEARCH》 (机器学习研究杂志)
年 卷 期:2021年第22卷第1期
页 面:1-83页
核心收录:
学科分类:08[工学] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术(可授工学、理学学位)]
主 题:Reinforcement Learning Approximate Dynamic Programming Approximate Policy Iteration Policy Oscillation Policy Chattering Markov Decision Process
摘 要:This paper presents a study of the policy improvement step that can be usefully exploited by approximate policy-iteration algorithms. When either the policy evaluation step or the policy improvement step returns an approximated result, the sequence of policies produced by policy iteration may not be monotonically increasing, and oscillations may occur. To address this issue, we consider safe policy improvements, i.e., at each iteration, we search for a policy that maximizes a lower bound to the policy improvement w.r.t. the current policy, until no improving policy can be found. We propose three safe policy-iteration schemas that differ in the way the next policy is chosen w.r.t. the estimated greedy policy. Besides being theoretically derived and discussed, the proposed algorithms are empirically evaluated and compared on some chain-walk domains, the prison domain, and on the Blackjack card game.