检索结果-内蒙古大学图书馆

Generalized dynamic programming principle and sparse mean-field control problems

JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS 2020年第1期481卷 123437-123437页

作者： Cavagnari, Giulia Marigonda, Antonio Piccoli, Benedetto Univ Pavia Dept Math F Casorati Via Ferrate 5 I-27100 Pavia Italy Univ Verona Dept Comp Sci Str Le Grazie 15 I-37134 Verona Italy Rutgers Univ Camden Dept Math Sci 311 N 5th St Camden NJ 08102 USA

In this paper we study optimal control problems in Wasserstein spaces, which are suitable to describe macroscopic dynamics of multi-particle systems. The dynamics is described by a parametrized continuity equation, in which the Eulerian velocity field is affine w.r.t. some variables. Our aim is to minimize a cost functional which includes a control norm, thus enforcing a control sparsity constraint. More precisely, we consider a nonlocal restriction on the total amount of control that can be used depending on the overall state of the evolving mass. We treat in details two main cases: an instantaneous constraint on the control applied to the evolving mass and a cumulative constraint, which depends also on the amount of control used in previous times. For both constraints, we prove the existence of optimal trajectories for general cost functions and that the value function is viscosity solution of a suitable Hamilton-Jacobi-Bellmann equation. Finally, we discuss an abstract dynamic programming principle, providing further applications in the Appendix. (C) 2019 Elsevier Inc. All rights reserved.

关键词： Multi-agent mean field sparse control Hamilton-Jacobi equation in Wasserstein space Control with uncertainty dynamic programming principle

来源：评论

学校读者我要写书评

暂无评论

On dynamic programming principle for Stochastic Control Under Expectation Constraints

引用

JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS 2020年第3期185卷 803-818页

作者： Chow, Yuk-Loong Yu, Xiang Zhou, Chao Sun Yat Sen Univ Sch Math Guangzhou Peoples R China Hong Kong Polytech Univ Dept Appl Math Hung Hom Kowloon Hong Kong Peoples R China Natl Univ Singapore Dept Math Singapore Singapore

This paper studies the dynamic programming principle using the measurable selection method for stochastic control of continuous processes. The novelty of this work is to incorporate intermediate expectation constraints on the canonical space at each time t. Motivated by some financial applications, we show that several types of dynamic trading constraints can be reformulated into expectation constraints on paths of controlled state processes. Our results can therefore be employed to recover the dynamic programming principle for these optimal investment problems under dynamic constraints, possibly path-dependent, in a non-Markovian framework.

关键词： dynamic programming principle Measurable selection Intermediate expectation constraints dynamic trading constraints

来源：评论

学校读者我要写书评

暂无评论

Feedback control of parametrized PDEs via model order reduction and dynamic programming principle

引用

ADVANCES IN COMPUTATIONAL MATHEMATICS 2020年第1期46卷 1-28页

作者： Alla, Alessandro Haasdonk, Bernard Schmidt, Andreas Pontificia Univ Catolica Rio de Janeiro Dept Math Rua Marques de Sao Vicente 225 BR-22453900 Rio de Janeiro Brazil Univ Stuttgart Inst Appl Anal & Numer Simulat Pfaffenwaldring 57 D-70569 Stuttgart Germany

In this paper, we investigate infinite horizon optimal control problems for parametrized partial differential equations. We are interested in feedback control via dynamic programming equations which is well-known to suffer from the curse of dimensionality. Thus, we apply parametric model order reduction techniques to construct low-dimensional subspaces with suitable information on the control problem, where the dynamic programming equations can be approximated. To guarantee a low number of basis functions, we combine recent basis generation methods and parameter partitioning techniques. Furthermore, we present a novel technique to construct non-uniform grids in the reduced domain, which is based on statistical information. Finally, we discuss numerical examples to illustrate the effectiveness of the proposed methods for PDEs in two space dimensions.

关键词： dynamic programming principle Semi-Lagrangian schemes Hamilton-Jacobi-Bellman equations Optimal control Model reduction Reduced basis method

来源：评论

学校读者我要写书评

暂无评论

dynamic programming principle AND ASSOCIATED HAMILTON-JACOBI-BELLMAN EQUATION FOR STOCHASTIC RECURSIVE CONTROL PROBLEM WITH NON-LIPSCHITZ AGGREGATOR

引用

ESAIM-CONTROL OPTIMISATION AND CALCULUS OF VARIATIONS 2018年第1期24卷 355-376页

作者： Pu, Jiangyan Zhang, Qi Shanghai Lixin Univ Accounting & Finance Sch Finance Shanghai 201209 Peoples R China Fudan Univ Sch Math Sci Shanghai 200433 Peoples R China

In this work we study the stochastic recursive control problem, in which the aggregator (or generator) of the backward stochastic differential equation describing the running cost is continuous but not necessarily Lipschitz with respect to the first unknown variable and the control, and monotonic with respect to the first unknown variable. The dynamic programming principle and the connection between the value function and the viscosity solution of the associated Hamilton-Jacobi-Bellman equation are established in this setting by the generalized comparison theorem for backward stochastic differential equations and the stability of viscosity solutions. Finally we take the control problem of continuous time Epstein Zin utility with non-Lipschitz aggregator as an example to demonstrate the application of our study.

关键词： Stochastic recursive control problem non-Lipschitz aggregator dynamic programming principle Hamilton-Jacobi-Bellman equation continuous-time Epstein Zin utility viscosity solution

来源：评论

学校读者我要写书评

暂无评论

Stochastic maximum principle, dynamic programming principle, and their relationship for fully coupled forward-backward stochastic controlled systems*

引用

ESAIM-CONTROL OPTIMISATION AND CALCULUS OF VARIATIONS 2020年第1期26卷 81-81页

作者： Hu, Mingshang Ji, Shaolin Xue, Xiaole Shandong Univ Zhongtai Secur Inst Financial Studies Jinan 250100 Shandong Peoples R China Shandong Univ Sch Management Jinan 250100 Peoples R China

Within the framework of viscosity solution, we study the relationship between the maximum principle (MP) from M. Hu, S. Ji and X. Xue [SIAM J. Control Optim. 56 (2018) 4309-4335] and the dynamic programming principle (DPP) from M. Hu, S. Ji and X. Xue [SIAM J. Control Optim. 57 (2019) 3911-3938] for a fully coupled forward-backward stochastic controlled system (FBSCS) with a nonconvex control domain. For a fully coupled FBSCS, both the corresponding MP and the corresponding Hamilton-Jacobi-Bellman (HJB) equation combine an algebra equation respectively. With the help of a new decoupling technique, we obtain the desirable estimates for the fully coupled forward-backward variational equations and establish the relationship. Furthermore, for the smooth case, we discover the connection between the derivatives of the solution to the algebra equation and some terms in the first-order and second-order adjoint equations. Finally, we study the local case under the monotonicity conditions as from J. Li and Q. Wei [SIAM J. Control Optim. 52 (2014) 1622-1662] and Z. Wu [Syst. Sci. Math. Sci. 11 (1998) 249-259], and obtain the relationship between the MP from Z. Wu [Syst. Sci. Math. Sci. 11 (1998) 249-259] and the DPP from J. Li and Q. Wei [SIAM J. Control Optim. 52 (2014) 1622-1662].

关键词： Fully coupled forward– backward stochastic differential equations global stochastic maximum principle dynamic programming principle viscosity solution monotonicity condition

来源：评论

学校读者我要写书评

暂无评论

dynamic programming principle for stochastic recursive optimal control problem driven by a G-Brownian motion

引用

STOCHASTIC PROCESSES AND THEIR APPLICATIONS 2017年第1期127卷 107-134页

作者： Hu, Mingshang ji, Shaolin Shandong Univ Zhongtai Secur Inst Financial Studies Jinan 250100 Shandong Peoples R China Shandong Univ Inst Math Jinan 250100 Peoples R China

In this paper, we study a stochastic recursive optimal control problem in which the cost functional is described by the solution of a backward stochastic differential equation driven by G-Brownian motion. Under standard assumptions, we establish the dynamic programming principle and the related fully nonlinear HJB equation in the framework of G-expectation. Finally, we show that the value function is the viscosity solution of the obtained RIB equation. (C) 2016 Elsevier B.V. All rights reserved.

关键词： G-expectation Backward stochastic differential equations Stochastic recursive optimal control Robust control dynamic programming principle

来源：评论

学校读者我要写书评

暂无评论

RANDOMIZED dynamic programming principle AND FEYNMAN-KAC REPRESENTATION FOR OPTIMAL CONTROL OF MCKEAN-VLASOV dynamicS

引用

TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY 2018年第3期370卷 2115-2160页

作者： Bayraktar, Erhan Cosso, Andrea Pham, Huyen Univ Michigan Dept Math 530 Church St Ann Arbor MI 48109 USA Politecn Milan Dipartimento Matemat Via Bonardi 9 I-20133 Milan Italy Univ Bologna Dipartimento Matemat Piazza Porta S Donato 5 I-40126 Bologna Italy Univ Paris Diderot Lab Probabilites & Modeles Aleatoires CNRS UMR 7599 F-75205 Paris 13 France Univ Paris Diderot Lab Probabilites & Modeles Aleatoires CNRS UMR 7599 Crest France

We analyze a stochastic optimal control problem, where the state process follows a McKean-Vlasov dynamics and the diffusion coefficient can be degenerate. We prove that its value function V admits a nonlinear FeynmanKac representation in terms of a class of forward-backward stochastic differential equations, with an autonomous forward process. We exploit this probabilistic representation to rigorously prove the dynamic programming principle (DPP) for V. The Feynman-Kac representation we obtain has an important role beyond its intermediary role in obtaining our main result: in fact it would be useful in developing probabilistic numerical schemes for V. The DPP is important in obtaining a characterization of the value function as a solution of a nonlinear partial differential equation (the so-called HamiltonJacobi-Belman equation), in this case on the Wasserstein space of measures. We should note that the usual way of solving these equations is through the Pontryagin maximum principle, which requires some convexity assumptions. There were attempts in using the dynamic programming approach before, but these works assumed a priori that the controls were of Markovian feedback type, which helps write the problem only in terms of the distribution of the state process (and the control problem becomes a deterministic problem). In this paper, we will consider open-loop controls and derive the dynamic programming principle in this most general case. In order to obtain the FeynmanKac representation and the randomized dynamic programming principle, we implement the so-called randomization method, which consists of formulating a new McKean-Vlasov control problem, expressed in weak form taking the supremum over a family of equivalent probability measures. One of the main results of the paper is the proof that this latter control problem has the same value function V of the original control problem.

关键词： Controlled McKean-Vlasov stochastic differential equations dynamic programming principle randomization method forward-backward stochastic differential equations

来源：评论

学校读者我要写书评

暂无评论

dynamic programming principle of control systems on manifolds and its relations to maximum principle

引用

JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS 2016年第1期434卷 915-938页

作者： Deng, Li Southwest Jiaotong Univ Sch Math Chengdu 610031 Peoples R China

We study the dynamic programming principle (DPP for short) on manifolds, obtain the Hamilton-Jacobi-Bellman (HJB for short) equation, and prove that the value function is the only viscosity solution to the HJB equation. Then, we investigate the relation between DPP and Pontryagin's maximum principle (PMP for short), from which we obtain PMP on manifolds. (C) 2015 Elsevier Inc. All rights reserved.

关键词： dynamic programming principle HJB equation Riemannian manifold Viscosity solution Pontryagin's maximum principle

来源：评论

学校读者我要写书评

暂无评论

dynamic programming principles for Mean-Field Controls with Learning

引用

OPERATIONS RESEARCH 2023年第4期71卷 1040-1054页

作者： Gu, Haotian Guo, Xin Wei, Xiaoli Xu, Renyuan Univ Calif Berkeley Dept Math Berkeley CA 94720 USA Univ Calif Berkeley Dept Ind Engn & Operat Res Berkeley CA 94720 USA Tsinghua Berkeley Shenzhen Inst Shenzhen 518055 Peoples R China Univ Southern Calif Ind & Syst Engn Los Angeles CA 90001 USA

The dynamic programming principle (DPP) is fundamental for control and optimization, including Markov decision problems (MDPs), reinforcement learning (RL), and, more recently, mean-field controls (MFCs). However, in the learning framework of MFCs, the DPP has not been rigorously established, despite its critical importance for algorithm designs. In this paper, we first present a simple example in MFCs with learning where the DPP fails with a misspecified Q function and then propose the correct form of Q function in an appropriate space for MFCs with learning. This particular form of Q function is different from the classical one and is called the IQ function. In the special case when the transition probability and the reward are independent of the mean-field information, it integrates the classical Q function for single-agent RL over the state-action distribution. In other words, MFCs with learning can be viewed as lifting the classical RLs by replacing the state-action space with its probability distribution space. This identification of the IQ function enables us to establish precisely the DPP in the learning framework of MFCs. Finally, we illustrate through numerical experiments the time consistency of this IQ function.

关键词： mean-field controls dynamic programming principle multi-agent reinforcement learning reinforcement learning Q-learning cooperative game

来源：评论

学校读者我要写书评

暂无评论

Relationship between maximum principle and dynamic programming principle for stochastic recursive optimal control problems of jump diffusions

引用

OPTIMAL CONTROL APPLICATIONS & METHODS 2014年第1期35卷 61-76页

作者： Shi, Jingtao Shandong Univ Sch Math Jinan 250100 Peoples R China

This paper is concerned with the relationship between maximum principle and dynamic programming principle for stochastic recursive optimal control problems of jump diffusions. Under the assumption that the value function is smooth, relations among the adjoint processes, the generalized Hamiltonian function, and the value function are given. A linear quadratic recursive utility portfolio optimization problem in the financial market is discussed to show the applications of the main result. Copyright (c) 2012 John Wiley & Sons, Ltd.

关键词： stochastic optimal control recursive utility backward stochastic differential equation jump diffusions maximum principle dynamic programming principle

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：