Data-driven adaptive dynamic programming for continuous-time fully cooperative games with partially constrained inputs

作     者：Zhang, Qichao Zhao, Dongbin Zhu, Yuanheng 

作者机构：Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Univ Chinese Acad Sci Beijing 100049 Peoples R China 

出 版 物：《NEUROCOMPUTING》 (神经计算)

年 卷 期：2017年第238卷

页      面：377-386页

核心收录：

学科分类：08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：National Natural Science Foundation of China (NSFC) [61273136, 61573353, 61533017, 61603382] National Key Research and Development Plan [2016YFB0101000] 

主　　题：Adaptive dynamic programming Optimal control Neural network Fully cooperative games Data-driven Constrained input 

摘      要：In this paper, the fully cooperative game with partially constrained inputs in the continuous-time Markov decision process environment is investigated using a novel data-driven adaptive dynamic programming method. First, the model-based policy iteration algorithm with one iteration loop is proposed, where the knowledge of system dynamics is required. Then, it is proved that the iteration sequences of value functions and control policies can converge to the optimal ones. In order to relax the exact knowledge of the system dynamics, a model-free iterative equation is derived based on the model-based algorithm and the integral reinforcement learning. Furthermore, a data-driven adaptive dynamic programming is developed to solve the model-free equation using generated system data. From the theoretical analysis, we prove that this model-free iterative equation is equivalent to the model-based iterative equations, which means that the data-driven algorithm can approach the optimal value function and control policies. For the implementation purpose, three neural networks are constructed to approximate the solution of the model-free iteration equation using the off-policy learning scheme after the available system data is collected in the online measurement phase. Finally, two examples are provided to demonstrate the effectiveness of the proposed scheme. (C) 2017 Published by Elsevier B.V.