检索结果-内蒙古大学图书馆

Has Dynamic Programming Improved Decision Making?

ANNUAL REVIEW OF ECONOMICS, VOL 11, 2019 2019年第1期11卷 833-858页

作者： Rust, John Georgetown Univ Dept Econ Washington DC 20057 USA

Dynamic programming (DP) is a powerful tool for solving a wide class of sequential decision-making problems under uncertainty. In principle, it enables us to compute optimal decision rules that specify the best possible decision in any situation. This article reviews developments in DP and contrasts its revolutionary impact on economics, operations research, engineering, and artificial intelligence with the comparative paucity of its real-world applications to improve the decision making of individuals and firms. The fuzziness of many real-world decision problems and the difficulty in mathematically modeling them are key obstacles to a wider application of DP in real-world settings. Nevertheless, I discuss several success stories, and I conclude that DP offers substantial promise for improving decision making if we let go of the empirically untenable assumption of unbounded rationality and confront the challenging decision problems faced every day by individuals and firms.

关键词： actor-critic algorithms dynamic programming artificial intelligence behavioral economics bounded rationality curse of dimensionality computational complexity neural networks revenue management reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Risk Averse Reinforcement Learning for Mixed Multi-Agent Environments 18

Risk Averse Reinforcement Learning for Mixed Multi-Agent Env...

引用

18th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS)

作者： Reddy, D. Sai Koti Saha, Amrita Tamilselvam, Srikanth G. Agrawal, Priyanka Dayama, Pankaj IBM Res Yorktown Hts NY 10598 USA

ISBN: (纸本)9781450363099

Most real world applications of multi-agent systems, need to keep a balance between maximizing the rewards and minimizing the risks. In this work we consider a popular risk measure, variance of return (VOR), as a constraint in the agent's policy learning algorithm in the mixed cooperative and competitive environments. We present a multi-timescale actor critic method for risk sensitive Markov games where the risk is modeled as a VOR constraint. We also show that the risk-averse policies satisfy the desired risk constraint without compromising much on the overall reward for a popular task.

关键词： Risk-Sensitive Reinforcement Learning Mixed Multi-Agent Environments actor-critic algorithms

来源：评论

学校读者我要写书评

暂无评论

Deep Reinforcement Learning for Market Making in Corporate Bonds: Beating the Curse of Dimensionality

引用

Applied Mathematical Finance 2019年第5期26卷 387-452页

作者： Guéant, Olivier Manziuk, Iuliia Centre d’Economie de la Sorbonne Université Paris 1 Panthéon-Sorbonne Paris France

In corporate bond markets, which are mainly OTC markets, market makers play a central role by providing bid and ask prices for bonds to asset managers. Determining the optimal bid and ask quotes that a market maker should set for a given universe of bonds is a complex task. The existing models, mostly inspired by the Avellaneda-Stoikov model, describe the complex optimization problem faced by market makers: proposing bid and ask prices for making money out of the difference between them while mitigating the market risk associated with holding inventory. While most of the models only tackle one-asset market making, they can often be generalized to a multi-asset framework. However, the problem of solving the equations characterizing the optimal bid and ask quotes numerically is seldom tackled in the literature, especially in high dimension. In this paper, we propose a numerical method for approximating the optimal bid and ask quotes over a large universe of bonds in a model à la Avellaneda–Stoikov. As classical finite difference methods cannot be used in high dimension, we present a discrete-time method inspired by reinforcement learning techniques, namely, a model-based deep actor-critic algorithm. © 2020, © 2020 Informa UK Limited, trading as Taylor & Francis Group.

关键词： actor-critic algorithms Market making reinforcement learning stochastic optimal control

来源：评论

学校读者我要写书评

暂无评论

SMART BUILDING REAL TIME PRICING FOR OFFERING LOAD-SIDE REGULATION SERVICE RESERVES

SMART BUILDING REAL TIME PRICING FOR OFFERING LOAD-SIDE REGU...

引用

52nd IEEE Annual Conference on Decision and Control (CDC)

作者： Bilgin, Enes Caramanis, Michael C. Paschalidis, Ioannis Ch. Boston Univ Coll Engn Ctr Informat & Syst Engn Boston MA 02215 USA

ISBN: (纸本)9781467357173

Provision of Regulation Service (RS) reserves to Power Markets by smart building demand response has attracted attention in recent literature. This paper develops tractable dynamic optimal pricing algorithms for distributed RS reserve provision. It shows monotonicity and convexity properties of the optimal pricing policies and the associated differential cost function. Then, it uses them to propose and implement a modified Least Squares Temporal Differences (LSTD) actor-critic algorithm with a bounded and continuous action space. This algorithm solves for the best policy within a pre-specified broad family. In addition, the paper develops a novel Approximate Policy Iteration (API) algorithm and uses it successfully to optimize the parameters of an analytic policy function. Numerical results are obtained to demonstrate and compare the actor-critic and Approximate Policy Iteration algorithms, demonstrating that the novel API algorithm outperforms the Bounded LSTD actor-critic algorithm in both computational effort and policy minimum cost.

关键词： Regulation Service Provision Distributed Demand Management actor-critic algorithms Approximate Policy Iteration

来源：评论

学校读者我要写书评

暂无评论

REINFORCEMENT LEARNING CONTROL FOR AUTOROTATION OF A SIMPLE POINT-MASS HELICOPTER MODEL

REINFORCEMENT LEARNING CONTROL FOR AUTOROTATION OF A SIMPLE ...

引用

作者： KADIRCAN KOPSA MIDDLE EAST TECHNICAL UNIVERSITY

学位级别：硕士

This study presents an application of an actor-critic reinforcement learning method to a simple point-mass model helicopter guidance problem during autorotation. A point-mass model of an OH-58A helicopter in autorotation was built. A reinforcement learning agent was trained by a model-free asynchronous actor-critic algorithm, where training episodes were parallelized on a multi-core CPU. Objective of the training was defined as achieving near-zero horizontal and vertical kinetic energies at the instant of touchdown. During each training episode, the agent was presented a reward at each discrete time-step according to a multi-conditional reward function. Reward function was programmed to output the negative of a weighted sum of squared vertical and horizontal velocities at touchdown. The agent consists of two separate neural network function approximators, namely the actor and the critic. The critic approx- imates the value of a set of states. The actor generates a set of actions given a set of states, sampled from a Gaussian distribution with mean values as output set of the actor network. Updates to the parameters of both networks were calculated from accumulated gradients during each episode and applied once per episode to improve training stability. RMSProp algorithm was used for optimization. Results achieved by the agent indicates that the method is successful at guiding the point-mass helicopter to the ground with minimal kinetic energy for most initial conditions. Controls generated by the reinforcement learning agent were found to be similar to a helicopter pilot's technique.

关键词： Reinforcement Learning Helicopter Autorotation actor-critic algorithms Artificial Neural Networks

来源：评论

学校读者我要写书评

暂无评论

Experimental evaluation of model-free reinforcement learning algorithms for continuous HVAC control

引用

APPLIED ENERGY 2021年 298卷 117164-117164页

作者： Biemann, Marco Scheller, Fabian Liu, Xiufeng Huang, Lizhen Tech Univ Denmark Dept Technol Management & Econ DK-2800 Lyngby Denmark Norwegian Univ Sci & Technol Dept Mfg & Civil Engn N-2815 Gjovik Norway

Controlling heating, ventilation and air-conditioning (HVAC) systems is crucial to improving demand-side energy efficiency. At the same time, the thermodynamics of buildings and uncertainties regarding human activities make effective management challenging. While the concept of model-free reinforcement learning demonstrates various advantages over existing strategies, the literature relies heavily on value-based methods that can hardly handle complex HVAC systems. This paper conducts experiments to evaluate four actor-critic algorithms in a simulated data centre. The performance evaluation is based on their ability to maintain thermal stability while increasing energy efficiency and on their adaptability to weather dynamics. Because of the enormous significance of practical use, special attention is paid to data efficiency. Compared to the model based controller implemented into EnergyPlus, all applied algorithms can reduce energy consumption by at least 10% by simultaneously keeping the hourly average temperature in the desired range. Robustness tests in terms of different reward functions and weather conditions verify these results. With increasing training, we also see a smaller trade-off between thermal stability and energy reduction. Thus, the Soft actor critic algorithm achieves a stable performance with ten times less data than on-policy methods. In this regard, we recommend using this algorithm in future experiments, due to both its interesting theoretical properties and its practical results.

关键词： Reinforcement learning Continuous HVAC control actor-critic algorithms Robustness Energy efficiency Soft actor critic

来源：评论

学校读者我要写书评

暂无评论

A Reinforcement Learning Based Algorithm for Finite Horizon Markov Decision Processes

A Reinforcement Learning Based Algorithm for Finite Horizon ...

引用

Proceedings of the 45th IEEE Conference on Decision and Control, Volume 1 of 14

作者： Shalabh Bhatnagar Mohammed Shahid Abdulla The Institute of Electrical and Electronics Engineers,Inc. Department of Computer Science and Automation Indian Institute of Science Bangalore INDIA

We develop a simulation based algorithm for finite horizon Markov decision processes with finite state and finite action space. Illustrative numerical experiments with the proposed algorithm are shown for problems in ... 详细信息

ISBN: (纸本)1424401704

关键词： Finite horizon Markov decision processes reinforcement learning two timescale stochastic approximation actor-critic algorithms normalized Hadamard matrices

来源：评论

学校读者我要写书评

暂无评论

Risk-sensitive reinforcement learning: a martingale approach to reward uncertainty 20

Risk-sensitive reinforcement learning: a martingale approach...

引用

Proceedings of the First ACM International Conference on AI in Finance

作者： Nelson Vadori Sumitra Ganesh Prashant Reddy Manuela Veloso J.P. Morgan AI Research

ISBN: (纸本)9781450375849

We introduce a novel framework to account for sensitivity to rewards uncertainty in sequential decision-making problems. While risk-sensitive formulations for Markov decision processes studied so far focus on the distribution of the cumulative reward as a whole, we aim at learning policies sensitive to the uncertain/stochastic nature of the rewards, which has the advantage of being conceptually more meaningful in some cases. To this end, we present a new decomposition of the randomness contained in the cumulative reward based on the Doob decomposition of a stochastic process, and introduce a new conceptual tool - the chaotic variation - which can rigorously be interpreted as the risk measure of the martingale component associated to the cumulative reward process. We innovate on the reinforcement learning side by incorporating this new risk-sensitive approach into model-free algorithms, both policy gradient and value function based, and illustrate its relevance on grid world and portfolio optimization problems.

关键词： policy gradient algorithms doob decomposition risk-sensitive martingale reward uncertainty risk-sensitive reinforcement learning actor-critic algorithms reinforcement learning markov decision process

来源：评论

学校读者我要写书评

暂无评论

Risk Averse Reinforcement Learning for Mixed Multi-agent Environments 19

Risk Averse Reinforcement Learning for Mixed Multi-agent Env...

引用

Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems

作者： D. Sai Koti Reddy Amrita Saha Srikanth G. Tamilselvam Priyanka Agrawal Pankaj Dayama IBM Research Bangalore India

ISBN: (纸本)9781450363099

关键词： mixed multi-agent environments actor-critic algorithms risk-sensitive reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Multi-Objective Reinforcement Learning with Non-Linear Scalarization 22

Multi-Objective Reinforcement Learning with Non-Linear Scala...

引用

Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems

作者： Mridul Agarwal Vaneet Aggarwal Tian Lan Purdue University West Lafayette IN USA George Washington University Washington DC USA

ISBN: (纸本)9781450392136

Multi-Objective Reinforcement Learning (MORL) setup naturally arises in many places where an agent optimizes multiple objectives. We consider the problem of MORL where multiple objectives are combined using a non-linear scalarization. We combine the vector objectives with a concave scalarization function and maximize this scalar objective. To work with the non-linear scalarization, in this paper, we propose a solution using steady-state occupancy measures and long-term average rewards. We show that when the scalarization function is element-wise increasing, the optimal policy for the scalarization is also Pareto optimal. To maximize the scalarized objective, we propose a model-based posterior sampling algorithm. Using a novel Bellman error analysis for infinite horizon MDPs based proof, we show that the proposed algorithm obtains a regret bound of ~O(LKDS√A/T) for K objectives, and L-Lipschitz continous scalarization function for MDP with S states, A actions, and diameter D. Additionally, we propose policy-gradient and actor-critic algorithms for MORL. For the policy gradient actor, we obtain the gradient using chain rule, and we learn different critics for each of the K objectives. Finally, we implement our algorithms on multiple environments including deep-sea treasure, and network scheduling setups to demonstrate that the proposed algorithms can optimize non-linear scalarization of multiple objectives.

关键词： regret analysis reinforcement learning actor-critic algorithms multi-objective reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：