Offline and Online Adaptive Critic Control Designs With Stability Guarantee Through Value Iteration

作     者：Ha, Mingming Wang, Ding Liu, Derong 

作者机构：Univ Sci & Technol Beijing Sch Automat & Elect Engn Beijing 100083 Peoples R China Beijing Univ Technol Fac Informat Technol Beijing 100124 Peoples R China Beijing Univ Technol Beijing Key Lab Computat Intelligence & Intellige Beijing 100124 Peoples R China Univ Illinois Dept Elect & Comp Engn Chicago IL 60607 USA 

出 版 物：《IEEE TRANSACTIONS ON CYBERNETICS》 (IEEE Trans. Cybern.)

年 卷 期：2022年第52卷第12期

页      面：13262-13274页

核心收录：

学科分类：0808[工学-电气工程] 08[工学] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：Beijing Natural Science Foundation [JQ19013] National Natural Science Foundation of China 

主　　题：Stability criteria Asymptotic stability Numerical stability Power system stability Heuristic algorithms Cost function Trajectory Adaptive dynamic programming asymptotic stability online adaptive critic control policy iteration (PI) reinforcement learning (RL) value iteration (VI) 

摘      要：This article is concerned with the stability of the closed-loop system using various control policies generated by value iteration. Some stability properties involving admissibility criteria, the attraction domain, and so forth, are investigated. An offline integrated value iteration (VI) scheme with a stability guarantee is developed by combining the advantages of VI and policy iteration, which is convenient to obtain admissible control policies. Also, based on the concept of attraction domain, an online adaptive dynamic programming algorithm using immature control policies is developed. Remarkably, it is ensured that the state trajectory under the online algorithm converges to the origin. Particularly, for linear systems, the online ADP algorithm with a general scheme possesses more enhanced stability property. The theoretical results reveal that the stability of the linear system can be guaranteed even if the control policy sequence includes finite unstable elements. The numerical results verify the effectiveness of the present algorithms.