检索结果-内蒙古大学图书馆

Stability and monotone convergence of generalised policy iteration for discrete-time linear quadratic regulations

INTERNATIONAL JOURNAL OF CONTROL 2016年第3期89卷 437-450页

作者： Chun, Tae Yoon Lee, Jae Young Park, Jin Bae Choi, Yoon Ho Yonsei Univ Dept Elect & Elect Engn Seoul 120749 South Korea Kyonggi Univ Dept Elect Engn Suwon Kyonggi Do South Korea

In this paper, we analyse the convergence and stability properties of generalised policy iteration (GPI) applied to discrete-time linear quadratic regulation problems. GPI is one kind of the generalised adaptive dynamic programming methods used for solving optimal control problems, and is composed of policy evaluation and policy improvement steps. To analyse the convergence and stability of GPI, the dynamic programming (DP) operator is defined. Then, GPI and its equivalent formulas are presented based on the notation of DP operator. The convergence of the approximate value function to the exact one in policy evaluation is proven based on the equivalent formulas. Furthermore, the positive semi-definiteness, stability, and the monotone convergence (PI-mode and VI-mode convergence) of GPI are presented under certain conditions on the initial value function. The online least square method is also presented for the implementation of GPI. Finally, some numerical simulations are carried out to verify the effectiveness of GPI as well as to further investigate the convergence and stability properties.

关键词： adaptive dynamic programming generalised policy iteration heuristic dynamic programming linear quadratic regulation

来源：评论

学校读者我要写书评

暂无评论

Approximate guaranteed cost fault-tolerant control of unknown nonlinear systems with time-varying actuator faults

引用

NONLINEAR dynamicS 2016年第1-2期83卷 269-282页

作者： Xie, Chun-Hua Yang, Guang-Hong Northeastern Univ Coll Informat Sci & Engn Shenyang 110819 Peoples R China Northeastern Univ State Key Lab Synthet Automat Proc Ind Shenyang 110819 Peoples R China

In this paper, the guaranteed cost fault-tolerant control problem for unknown multi-input continuous nonlinear systems with loss of actuator effectiveness faults is investigated using the adaptive dynamic programming algorithm. Initially, by modifying the cost function to account for actuator faults, the problem is transformed into an optimal control problem of the nominal system. Subsequently, by using an existing policy iteration (PI) algorithm to solve the corresponding optimal control problem, a guaranteed cost controller is constructed approximately. Furthermore, a rigorous proof is given to show the convergence of the aforementioned PI algorithm while taking the neural network approximation errors into consideration. Finally, simulation examples are provided to show the effectiveness of the proposed approach.

关键词： Unknown nonlinear systems adaptive dynamic programming Fault-tolerant control Policy iteration Neural network

来源：评论

学校读者我要写书评

暂无评论

Using reinforcement learning techniques to solve continuous-time non-linear optimal tracking problem without system dynamics

引用

IET CONTROL THEORY AND APPLICATIONS 2016年第12期10卷 1339-1347页

作者： Zhu, Yuanheng Zhao, Dongbin Li, Xiangjun Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China China Elect Power Res Inst Elect Engn & New Mat Dept Beijing 100192 Peoples R China

The optimal tracking of non-linear systems without knowing system dynamics is an important and intractable problem. Based on the framework of reinforcement learning (RL) and adaptive dynamic programming, a model-free adaptive optimal tracking algorithm is proposed in this study. After constructing an augmented system with the tracking errors and the reference states, the tracking problem is converted to a regulation problem with respect to the new system. Several RL techniques are synthesised to form a novel algorithm which learns the optimal solution online in real time without any information of the system dynamics. Continuous adaptation laws are defined by the current observations and the past experience. The convergence is guaranteed by Lyapunov analysis. Two simulations on a linear and a non-linear systems demonstrate the performance of the proposed approach.

关键词： nonlinear control systems continuous time systems learning (artificial intelligence) optimal control dynamic programming Lyapunov methods linear systems reinforcement learning continuous-time problem nonlinear optimal tracking problem adaptive dynamic programming model-free adaptive optimal tracking algorithm Lyapunov analysis linear system

来源：评论

学校读者我要写书评

暂无评论

A neural-network-based online optimal control approach for nonlinear robust decentralized stabilization

引用

SOFT COMPUTING 2016年第2期20卷 707-716页

作者： Wang, Ding Liu, Derong Li, Hongliang Ma, Hongwen Li, Chao Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China

In this paper, the robust decentralized stabilization of continuous-time uncertain nonlinear systems with multi control stations is developed using a neural network based online optimal control approach. The novelty lies in that the well-known adaptive dynamic programming method is extended to deal with the nonlinear feedback control problem under uncertain and large-scale environment. Through introducing an appropriate bounded function and defining a modified cost function, it can be observed that the decentralized optimal controller of the nominal system can achieve robust decentralized stabilization of original uncertain system. Then, a critic neural network is constructed for solving the modified Hamilton-Jacobi-Bellman equation corresponding to the nominal system in an online fashion. The weights of the critic network are tuned based on the standard steepest descent algorithm with an additional term provided to guarantee the boundedness of system states. The stability analysis of the closed-loop system is carried out via the Lyapunov approach. At last, two simulation examples are given to verify the effectiveness of the present control approach.

关键词： adaptive dynamic programming Approximate dynamic programming Neural networks Online optimal control Robust decentralized stabilization Uncertain nonlinear systems

来源：评论

学校读者我要写书评

暂无评论

Data-based robust optimal control of continuous-time affine nonlinear systems with matched uncertainties

引用

INFORMATION SCIENCES 2016年 366卷 121-133页

作者： Wang, Ding Li, Chao Liu, Derong Mu, Chaoxu Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Univ Sci & Technol Beijing Sch Automat & Elect Engn Beijing 100083 Peoples R China Tianjin Univ Sch Elect Engn & Automat Tianjin Key Lab Proc Measurement & Control Tianjin 300072 Peoples R China

In this paper, the robust optimal control of continuous-time affine nonlinear systems with matched uncertainties is investigated by using a data-based integral policy iteration approach. It is a natural extension of the traditional optimal control design, under the framework of adaptive dynamic programming (ADP) method, to robust optimal control of nonlinear systems with matched uncertainties. In theoretical aspect, by increasing a feedback gain to the optimal controller of the nominal system, the robust controller of the matched uncertain system is obtained, which also achieves optimality with a newly well-defined cost function. When regarding the implementation, the data-based integral policy iteration algorithm is used to solve the Hamilton-Jacobi-Bellman equation corresponding to the nominal system with completely unknown dynamics information. Then, the actor-critic technique based on neural networks and least squares implementation method are employed to facilitate deriving the optimal control law iteratively, so that the closed-form expression of the robust optimal controller is available. Additionally, two simulation examples with application backgrounds are presented to illustrate the effectiveness of the established robust optimal control scheme. In summary, it is important to note that the result developed in this paper broadens the application scope of ADP-based optimal control approach to more general nonlinear systems possessing dynamical uncertainties. (C) 2016 Elsevier Inc. All rights reserved.

关键词： adaptive dynamic programming Data-based control Integral policy iteration Matched uncertainties Neural networks Robust optimal control

来源：评论

学校读者我要写书评

暂无评论

Decentralized guaranteed cost control of interconnected systems with uncertainties: A learning-based optimal control strategy

引用

NEUROCOMPUTING 2016年 214卷 297-306页

作者： Wang, Ding Liu, Derong Mu, Chaoxu Ma, Hongwen Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Univ Sci & Technol Beijing Sch Automat & Elect Engn Beijing 100083 Peoples R China Tianjin Univ Sch Elect Engn & Automat Tianjin Key Lab Proc Measurement & Control Tianjin 300072 Peoples R China

A novel learning-based optimal control approach is constructed to attain the decentralized guaranteed cost controller design for a class of continuous-time complex nonlinear systems with dynamical uncertainties and interconnections. This is performed by combining robust decentralized control formulation with adaptive critic learning technique. By expressing the interconnected subsystems as a whole system and introducing a new cost function for the overall plant, the decentralized guaranteed cost control problem is formulated as an optimal control problem for the nominal overall system. Then, a policy iteration based learning control algorithm is employed to solve the modified Hamilton-Jacobi-Bellman equation with respect to the nominal plant iteratively. A critic neural network is constructed to approximate the optimal state feedback control law and then the uniform ultimate boundedness stability issue is analyzed. Meanwhile, a simulation experiment is conducted to verify the good performance of the control approach. (C) 2016 Elsevier B.V. All rights reserved.

关键词： adaptive dynamic programming Decentralized control Guaranteed cost control Interconnected systems Learning control Neural networks Optimal control Uncertain plant

来源：评论

学校读者我要写书评

暂无评论

Optimal tracking control for completely unknown nonlinear discrete-time Markov jump systems using data-based reinforcement learning method

引用

NEUROCOMPUTING 2016年 194卷 176-182页

作者： Jiang, He Zhang, Huaguang Luo, Yanhong Wang, Junyi Northeastern Univ Coll Informat Sci & Engn Box 134 Shenyang 110819 Peoples R China

In this paper, we develop a novel optimal tracking control scheme for a class of nonlinear discrete-time Markov jump systems (MJSs) by utilizing a data-based reinforcement learning method. It is not practical to obtain accurate system models of the real-world MJSs due to the existence of abrupt variations in their system structures. Consequently, most traditional model-based methods for MJSs are invalid for the practical engineering applications. In order to overcome the difficulties without any identification scheme which would cause estimation errors, a model-free adaptive dynamic programming (ADP) algorithm will be designed by using system data rather than accurate system functions. Firstly, we combine the tracking error dynamics and reference system dynamics to form an augmented system. Then, based on the augmented system, a new performance index function with discount factor is formulated for the optimal tracking control problem via Markov chain and weighted sum technique. Neural networks are employed to implement the on-line ADP learning algorithm. Finally, a simulation example is given to demonstrate the effectiveness of our proposed approach. (C) 2016 Elsevier B.V. All rights reserved.

关键词： Optimal tracking control Markov jump systems Data-based Reinforcement learning adaptive dynamic programming Neural networks

来源：评论

学校读者我要写书评

暂无评论

Improvement of LMI controllers of Takagi-Sugeno models via Q-learning

引用

IFAC-PapersOnLine 2016年第5期49卷 67-72页

作者： Díaz, Henry Armesto, Leopoldo Sala, Antonio Universitat Politècnica de València C/Camino de Vera s/n Valencia46022 Spain

This paper presents a preliminary attempt to bridge the conservative (shape-independent) results from guaranteed-cost LMIs and the reinforcement learning setups which learn optimal controllers from data. In this sense, the proposed approach uses an initialization based on the LMI solution and proposes an approximation of the Q-function using polynomials of the membership functions in Takagi-Sugeno models. The resulting controller is shape-dependent, that is, uses the knowledge of membership functions and data to clearly improve LMI solutions. © 2016

关键词： Reinforcement learning Controllers dynamic programming Linear matrix inequalities Membership functions Modal analysis Polynomials adaptive dynamic programming Guaranteed cost Lmi solutions Optimal controller Q functions Q learning Takagi sugeno Takagi Sugeno models

来源：评论

学校读者我要写书评

暂无评论

Mobile Robot Control Based on Hybrid Neuro-Fuzzy Value Gradient Reinforcement Learning

Mobile Robot Control Based on Hybrid Neuro-Fuzzy Value Gradi...

引用

International Joint Conference on Neural Networks

作者： Seaar Al-Dabooni Donald Wunsch Dept. of Electrical and Computer Engineering Missouri University of Science and Technology

ISBN: (纸本)9781509061839

This paper uses value gradient learning (VGL) to track a reference trajectory under uncertainties, by computing the optimal left and right torque values for a nonholonomic mobile robot. VGL is a high-performance algorithm in adaptive dynamic programming (ADP). Here, it is used as a critic function after fitting a first-order Sugeno fuzzy neural network (FNN) structure to critic and actor networks. Moreover, this work handles the impacts of unmodeled bounded disturbances with various friction values. The simulation is introduced to compare two approaches. The first uses an actor network that confirms the ability of the mobile robot dynamic model to follow a desired trajectory. This approach demonstrates a significant enhancement of the robot's capability to absorb unstructured disturbance signals and friction effects. The second type of results use a critic-optimal-control approach, calculating the optimal control signal for the affine dynamic model of the robot. This completely removes the actor network to exploit reduced computational complexity with faster responses. The simulation is introduced to compare both cases.

关键词： Nonholonomic dynamic mobile robot Fuzzy neural network adaptive dynamic programming Fuzzy neural networks Robot control dynamic programming critic HDLBP gene actor Mobile robots Learning learning (artificial intelligence)

来源：评论

学校读者我要写书评

暂无评论

Heuristic dynamic programming Using Echo State Network For Multivariable Tracking Control Of Wastewater Treatment Process

引用

ASIAN JOURNAL OF CONTROL 2015年第5期17卷 1654-1666页

作者： Bo Ying-Chun Qiao Jun-Fei China Univ Petr Coll Informat & Control Engn Qingdao 266580 Peoples R China Beijing Univ Technol Coll Elect & Control Engn Beijing 100124 Peoples R China

This paper proposes a heuristic dynamic programming (HDP) scheme to simultaneously control the dissolved oxygen concentration and the nitrate level in wastewater treatment processes (WWTP). Unlike traditional HDP schemes, the optimal control values are calculated in an analytical way by the proposed HDP controller. It can reduce the learning burden of the HDP controller to a great extent. The system model and the evaluation index J are approximated by two echo state networks (ESNs). Gradient-based learning algorithms are employed to train both ESNs online, and the convergence of the training algorithm is investigated based on Lyapunov theory. The performance of the proposed ESN-based HDP (E-HDP) controller is tested and evaluated on a WWTP benchmark. Experimental results demonstrate that the proposed approach can achieve effective performance.

关键词： adaptive dynamic programming quadratic performance index echo state networks convergence wastewater treatment

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：