检索结果-内蒙古大学图书馆

Policy iteration based Q-learning for linear nonzero-sum quadratic differential games

Science China(Information Sciences) 2019年第5期62卷 195-213页

作者： Xinxing LI Zhihong PENG Li LIANG Wenzhong ZHA School of Automation Beijing Institute of Technology State Key Laboratory of Intelligent Control and Decision of Complex System Information Science Academy China Electronics Technology Group Corporation

In this paper, a policy iteration-based Q-learning algorithm is proposed to solve infinite horizon linear nonzero-sum quadratic differential games with completely unknown dynamics. The Q-learning algorithm, which employs off-policy reinforcement learning(RL), can learn the Nash equilibrium and the corresponding value functions online, using the data sets generated by behavior policies. First, we prove equivalence between the proposed off-policy Q-learning algorithm and an offline PI algorithm by selecting specific initially admissible polices that can be learned online. Then, the convergence of the off-policy Qlearning algorithm is proved under a mild rank condition that can be easily met by injecting appropriate probing noises into behavior policies. The generated data sets can be repeatedly used during the learning process, which is computationally effective. The simulation results demonstrate the effectiveness of the proposed Q-learning algorithm.

关键词： adaptive dynamic programming ADP Q-learning reinforcement learning RL linear nonzerosum quadratic differential games policy iteration PI off-policy

来源：评论

学校读者我要写书评

暂无评论

Observer-based fuzzy adaptive optimal control for nonlinear continuous-time saturation interconnected systems

引用

OPTIMAL CONTROL APPLICATIONS & METHODS 2018年第4期39卷 1273-1290页

作者： Wang, Tiechao Tong, Shaocheng Liaoning Univ Technol Coll Elect Engn Jinzhou 121001 Peoples R China

A fuzzy adaptive optimal output-feedback control problem is investigated for nonlinear continuous-time interconnected systems with saturation constraints. The system dynamics and the mismatched interconnections of each subsystem are unknown, and each subsystem contains unavailable states. Based on fuzzy logic systems, a fuzzy state observer is designed to approximate the unknown system states and the interconnections of the nonlinear interconnected system. By using adaptive dynamic programming technology, the decentralized optimal controller of each subsystem is given and the saturation control input is updated via the critic fuzzy logic systems. The proposed fuzzy adaptive observer-based optimal control scheme can guarantee that the closed-loop interconnected system is ultimately uniformly bounded stable. A simulation example validates the effectiveness of the presented scheme.

关键词： adaptive dynamic programming fuzzy logic systems saturation constraint state observers

来源：评论

学校读者我要写书评

暂无评论

Reinforcement learning for control: Performance, stability, and deep approximators

引用

ANNUAL REVIEWS IN CONTROL 2018年 46卷 8-28页

作者： Busoniu, Lucian de Bruin, Tim Tolic, Domagoj Kober, Jens Palunko, Ivana Tech Univ Cluj Napoca Cluj Napoca Romania Delft Univ Technol Delft Netherlands RIT Croatia Don Frana Bulica 6 Dubrovnik 20000 Croatia Univ Dubrovnik Cira Carica 4 Dubrovnik 20000 Croatia

Reinforcement learning (RL) offers powerful algorithms to search for optimal controllers of systems with nonlinear, possibly stochastic dynamics that are unknown or highly uncertain. This review mainly covers artificial-intelligence approaches to RL, from the viewpoint of the control engineer. We explain how approximate representations of the solution make RL feasible for problems with continuous states and control actions. Stability is a central concern in control, and we argue that while the control-theoretic RL subfield called adaptive dynamic programming is dedicated to it, stability of RL largely remains an open question. We also cover in detail the case where deep neural networks are used for approximation, leading to the field of deep RL, which has shown great success in recent years. With the control practitioner in mind, we outline opportunities and pitfalls of deep RL;and we close the survey with an outlook that - among other things - points out some avenues for bridging the gap between control and artificial-intelligence RL techniques. (C) 2018 Elsevier Ltd. All rights reserved.

关键词： Reinforcement learning Optimal control Deep learning Stability Function approximation adaptive dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Learning-based adaptive prescribed performance control of postcapture space robot-target combination without inertia identifications

引用

ACTA ASTRONAUTICA 2018年 146卷 228-242页

作者： Wei, Caisheng Luo, Jianjun Dai, Honghua Bian, Zilin Yuan, Jianping Northwestern Polytech Univ Natl Key Lab Aerosp Flight Dynam Xian 710072 Shaanxi Peoples R China NanChang HangKong Univ Sch Informat Engn Nanchang 330063 Jiangxi Peoples R China

In this paper, a novel learning-based adaptive attitude takeover control method is investigated for the post capture space robot-target combination with guaranteed prescribed performance in the presence of unknown inertial properties and external disturbance. First, a new static prescribed performance controller is developed to guarantee that all the involved attitude tracking errors are uniformly ultimately bounded by quantitatively characterizing the transient and steady-state performance of the combination. Then, a learning-based supplementary adaptive strategy based on adaptive dynamic programming is introduced to improve the tracking performance of static controller in terms of robustness and adaptiveness only utilizing the input/output data of the combination. Compared with the existing works, the prominent advantage is that the unknown inertial properties are not required to identify in the development of learning-based adaptive control law, which dramatically decreases the complexity and difficulty of the relevant controller design. Moreover, the transient and steady-state performance is guaranteed a priori by designer-specialized performance functions without resorting to repeated regulations of the controller parameters. Finally, the three groups of illustrative examples are employed to verify the effectiveness of the proposed control method.

关键词： Prescribed performance Spacecraft adaptive dynamic programming Nonlinear control Space robot

来源：评论

学校读者我要写书评

暂无评论

Policy Iteration Algorithm Based Fault Tolerant Tracking Control: An Implementation on Reconfigurable Manipulators

引用

JOURNAL OF ELECTRICAL ENGINEERING & TECHNOLOGY 2018年第4期13卷 1739-1750页

作者： Li, Yuanchun Xia, Hongbing Zhao, Bo Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Changchun Univ Technol Dept Control Sci & Engn Changchun 130012 Jilin Peoples R China Hytera Commun Co Ltd Nanjing 210000 Jiangsu Peoples R China

This paper proposes a novel fault tolerant tracking control (FTTC) scheme for a class of nonlinear systems with actuator failures based on the policy iteration (PI) algorithm and the adaptive fault observer. The estimated actuator failure from an adaptive fault observer is utilized to construct an improved performance index function that reflects the failure, regulation and control simultaneously. With the help of the proper performance index function, the FTTC problem can be transformed into an optimal control problem. The fault tolerant tracking controller is composed of the desired controller and the approximated optimal feedback one. The desired controller is developed to maintain the desired tracking performance at the steady-state, and the approximated optimal feedback controller is designed to stabilize the tracking error dynamics in an optimal manner. By establishing a critic neural network, the PI algorithm is utilized to solve the Hamilton-Jacobi-Bellman equation, and then the approximated optimal feedback controller can be derived. Based on Lyapunov technique, the uniform ultimate boundedness of the closed-loop system is proven. The proposed FTTC scheme is applied to reconfigurable manipulators with two degree of freedoms in order to test the effectiveness via numerical simulation.

关键词： adaptive dynamic programming Policy iteration Fault tolerant tracking control Reconfigurable manipulators Neural network

来源：评论

学校读者我要写书评

暂无评论

Learning and Guaranteed Cost Control With Event-Based adaptive Critic Implementation

引用

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2018年第12期29卷 6004-6014页

作者： Wang, Ding Liu, Derong Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Univ Chinese Acad Sci Sch Comp & Control Engn Beijing 100049 Peoples R China Guangdong Univ Technol Sch Automat Guangzhou 510006 Guangdong Peoples R China

This paper focuses on the event-triggered guaranteed cost control design of nonlinear systems via a self-learning technique. In brief, an event-based guaranteed cost control strategy of nonlinear systems subjects to matched uncertainties is developed, thereby balancing the performance of guaranteed cost and the actuality of limited communication resource. The original control design is transformed into an optimal control problem with an event-based mechanism, where the relationship of guaranteed cost performance compared to the time-based formulation is discussed. A critic neural network is constructed for implementing the event-based optimal control design with stability guarantee. Simulation experiments are carried out to verify the theoretical results in detail.

关键词： adaptive dynamic programming event-based design guaranteed cost control optimal control self-learning technique

来源：评论

学校读者我要写书评

暂无评论

Reinforcement learning for robust adaptive control of partially unknown nonlinear systems subject to unmatched uncertainties

引用

INFORMATION SCIENCES 2018年 463卷 307-322页

作者： Yang, Xiong He, Haibo Wei, Qinglai Luo, Biao Tianjin Univ Sch Elect & Informat Engn Tianjin 300072 Peoples R China Univ Rhode Isl Dept Elect Comp & Biomed Engn Kingston RI 02881 USA Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China

This paper proposes a novel robust adaptive control strategy for partially unknown continuous-time nonlinear systems subject to unmatched uncertainties. Initially, the robust nonlinear control problem is converted into a nonlinear optimal control problem by constructing an appropriate value function for the auxiliary system. After that, within the framework of reinforcement learning, an identifier-critic architecture is developed. The presented architecture uses two neural networks: the identifier neural network (INN) which aims at estimating the unknown internal dynamics and the critic neural network (CNN) which tends to derive the approximate solution of the Hamilton-jacobi-Bellman equation arising in the obtained optimal control problem. The INN is updated by using both the back-propagation algorithm and the e-modification technique. Meanwhile, the CNN is updated via the modified gradient descent method, which uses historical and current state data simultaneously. Based on the classic Lyapunov technique, all the signals in the closed-loop auxiliary system are proved to be uniformly ultimately bounded. Moreover, the original system is kept asymptotically stable under the obtained approximate optimal control. Finally, two illustrative examples, including the F-16 aircraft plant, are provided to demonstrate the effectiveness of the developed method. (C) 2018 Elsevier Inc. All rights reserved.

关键词： adaptive dynamic programming Neural networks Optimal control Reinforcement learning Robust control Unmatched uncertainty

来源：评论

学校读者我要写书评

暂无评论

Stability Analysis of Optimal adaptive Control Using Value Iteration With Approximation Errors

引用

IEEE TRANSACTIONS ON AUTOMATIC CONTROL 2018年第9期63卷 3119-3126页

作者： Heydari, Ali Southern Methodist Univ Dept Mech Engn Dallas TX 75205 USA

Effects of the presence of approximation errors are analyzed on the stability of adaptive optimal control using value iteration, initiated from a stabilizing control policy. This analysis includes the system operated using any single/constant resulting control policy and also using an evolving/time-varying control policy. Sufficient conditions on the 'per iteration' approximation errors are obtained for guaranteeing the stability. A feature of the presented results is providing estimations of the region of attraction, under the approximation errors, so that if the initial condition is within this region, the whole trajectory will remain inside the training region, and hence, the function approximation results remain reliable.

关键词： adaptive dynamic programming approximation error stability analysis value iteration

来源：评论

学校读者我要写书评

暂无评论

Online optimal solutions for multi-player nonzero-sum game with completely unknown dynamics

引用

NEUROCOMPUTING 2018年 283卷 87-97页

作者： Lv, Yongfeng Ren, Xuemei Na, Jing Beijing Inst Technol Sch Automat Beijing 100081 Peoples R China Kunming Univ Sci & Technol Fac Mech & Elect Engn Kunming 650500 Yunnan Peoples R China

In this paper, a data-driven approximate neural network (NN) learning scheme is developed to solve the multi-player nonzero-sum (NZS) game problem with completely unknown system dynamics. An augmented NN identifier based on a new parameter estimation algorithm is first established to approximate the completely unknown system dynamics. Then approximated dynamic programming (ADP) with neural networks is constructed to approximate the optimal solutions of the coupled Hamilton-Jacobi equations for each player. The approximated NN value functions are then used to synchronously calculate the optimal control policies for every player. The identifier and ADP NN weights are online updated with the system input-output data based on a novel adaptive law, which could achieve a faster convergence speed. Moreover, the convergence of all NN weights and the stability of the closed-loop system are proved based on the Lyapunov approach. Finally, a dual-driven servo motor system and a three-player nonlinear game system are simulated to verify the feasibility of the developed methods. (C) 2017 Elsevier B.V. All rights reserved.

关键词： Neural networks adaptive dynamic programming System identification Optimal control Multi-player nonzero-sum games

来源：评论

学校读者我要写书评

暂无评论

Distributed optimal control for nonholonomic systems with input constraints and uncertain interconnections

引用

NONLINEAR dynamicS 2018年第2期93卷 801-817页

作者： Luy Nguyen Tan Ind Univ Ho Chi Minh City 12 Nguyen Van Bao St Ho Chi Minh City Vietnam

This paper studies a distributed optimal tracking control method for nonholonomic mobile mechanical multi-agent systems under complex conditions such as input constraints, the presence of both kinematic and dynamic disturbances, and uncertain interconnections. Initially, novel feed-forward control inputs are proposed to transform the inherently separate systems of kinematics and dynamics into an equivalent integrated system. Successively, an online distributed -bounded optimal control algorithm is designed by utilizing adaptive dynamic programming and the theory of cooperative differential graphical games. In the algorithm, a single neural network instead of three for each agent is chosen, and the online weight-tuning laws for which are designed without identifying uncertain parameters directly or indirectly. Additionally, the optimal control and worst disturbance policies are synchronously updated in only one iterative loop. It is shown that during the convergence of the value functions to the approximate optimal values when the agents perform the algorithm, overall tracking and function approximation errors are uniformly ultimately bounded. Finally, as a successful application of the study, control of the wheeled mobile multi-robot system is discussed through simulations.

关键词： Nonholonomic systems Distributed optimal control Input constrains adaptive dynamic programming Mobile robot systems Disturbance

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：