检索结果-内蒙古大学图书馆

Robust control scheme for a class of uncertain nonlinear systems with completely unknown dynamics using data-driven reinforcement learning method

引用

NEUROCOMPUTING 2018年 273卷 68-77页

作者： Jiang, He Zhang, Huaguang Cui, Yang Xiao, Geyang Northeastern Univ Coll Informat Sci & Engn Box 134 Shenyang 110819 Liaoning Peoples R China

This paper deals with the robust control issues for a class of uncertain nonlinear systems with completely unknown dynamics via a data-driven reinforcement learning method. Firstly, we formulate the optimal regulation control problem for the nominal system, and then, the robust controller for the original uncertain system is designed by adding a constant feedback gain to the optimal controller of the nominal system. Then, this scheme is extended to the optimal tracking control by means of augmented system and discount factor. It is also demonstrated that the proposed robust controller can achieve optimality with a new defined performance index function when there is no control perturbation. It is well known that the nonlinear optimal control problem relies on the solution of Hamilton-Jacobi-Bellman (HJB) equation, which is a nonlinear partial differential equation and impossible to be solved analytically. In order to overcome this difficulty, we introduce a model-based iterative learning algorithm to successively approximate the solution of HJB equation and provide its convergence proof. Subsequently, based on the structure of the model-based approach, a data-driven reinforcement learning method is derived, which only requires the sampling data from real system with different control inputs rather than the accurate mathematical system models. Neural networks (NNs) are utilized to implement this model-free method to approximate the optimal solutions and the least-square approach is employed to minimize the NN approximation residual errors. Finally, two numerical simulation examples are given to illustrate the effectiveness of our proposed method. (C) 2017 Published by Elsevier B.V.

关键词： Reinforcement learning adaptive dynamic programming Data-driven Model-free Neural networks

来源：评论

学校读者我要写书评

暂无评论

Neural robust stabilization via event-triggering mechanism and adaptive learning technique

引用

NEURAL NETWORKS 2018年 102卷 27-35页

作者： Wang, Ding Liu, Derong Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Univ Chinese Acad Sci Sch Comp & Control Engn Beijing 100049 Peoples R China Guangdong Univ Technol Sch Automat Guangzhou 510006 Guangdong Peoples R China

The robust control synthesis of continuous-time nonlinear systems with uncertain term is investigated via event-triggering mechanism and adaptive critic learning technique. We mainly focus on combining the event-triggering mechanism with adaptive critic designs, so as to solve the nonlinear robust control problem. This can not only make better use of computation and communication resources, but also conduct controller design from the view of intelligent optimization. Through theoretical analysis, the nonlinear robust stabilization can be achieved by obtaining an event-triggered optimal control law of the nominal system with a newly defined cost function and a certain triggering condition. The adaptive critic technique is employed to facilitate the event-triggered control design, where a neural network is introduced as an approximator of the learning phase. The performance of the event-triggered robust control scheme is validated via simulation studies and comparisons. The present method extends the application domain of both event-triggered control and adaptive critic control to nonlinear systems possessing dynamical uncertainties. (c) 2018 Elsevier Ltd. All rights reserved.

关键词： adaptive critic designs adaptive dynamic programming Event-triggered control Neural networks Robust stabilization Uncertainties

来源：评论

学校读者我要写书评

暂无评论

Decentralized reliable guaranteed cost control for large-scale nonlinear systems using actor-critic network

引用

NEUROCOMPUTING 2018年 320卷 121-128页

作者： Ye, Dan Song, Tingting Northeastern Univ Coll Informat Sci & Engn Shenyang 110819 Liaoning Peoples R China Northeastern Univ State Key Lab Synthet Automat Proc Ind Shenyang 110189 Liaoning Peoples R China

In this paper, the decentralized reliable guaranteed cost control problem of large-scale nonlinear systems with mismatched interconnection and time-varying actuator fault is investigated. The unknown actuator fault is assumed to be with known upper bounds. Under the case that the interconnection term is unknown and mismatched, the guaranteed cost control problem for a class of large-scale nonlinear systems can be transformed into design an optimal control strategy of the isolated system, which is necessary to solve the HJB (Hamilton-Jacobi-Bellman) equations by using adaptive dynamic programming (ADP) of actor-critic networks. The convergence property of the neural network weights and the stability of the closed-loop system have been strictly proved even if time-varying actuator faults and unknown interconnection coexist. Finally, simulation examples are given to show the effectiveness of the proposed approach. (C) 2018 Elsevier B.V. All rights reserved.

关键词： Large-scale nonlinear systems Actor-critic networks Reliable control adaptive dynamic programming Guaranteed cost control Mismatched interconnection

来源：评论

学校读者我要写书评

暂无评论

Linear quadratic tracking control of unknown discrete-time systems using value iteration algorithm

引用

NEUROCOMPUTING 2018年 314卷 86-93页

作者： Li, Xiaofeng Xue, Lei Sun, Changyin Southeast Univ Sch Automat Nanjing 210096 Jiangsu Peoples R China

In this paper, an optimal tracking control scheme is proposed to solve the infinite-horizon linear quadratic tracking (LQT) problem using iterative adaptive dynamic programming (ADP) algorithm. The reference trajectory is assumed to be produced by a linear command generator. First, via system transformation, an augmented system composed of controlled system and command generator is constructed. Then we derive the Bellman equation in terms of the transformed system with discount factor in cost function. In order to avoid requirement for knowledge of system dynamics, the iterative ADP algorithm is introduced to solve the Bellman equation with convergence analysis. A novel approach based on controllability and observability analysis is presented to show the stability of tracking error. For facilitating the implementation of this iterative approach, three neural networks (NNs) are employed as parametric structures to identify the unknown system dynamics, approximate performance function and search control policy, respectively. Finally, a simulation example is included to verify the effectiveness of the proposed scheme. (C) 2018 Published by Elsevier B.V.

关键词： adaptive dynamic programming Linear quadratic tracking Reinforcement learning Value iteration

来源：评论

学校读者我要写书评

暂无评论

Observer-Based adaptive Fuzzy Decentralized Optimal Control Design for Strict-Feedback Nonlinear Large-Scale Systems

引用

IEEE TRANSACTIONS ON FUZZY SYSTEMS 2018年第2期26卷 569-584页

作者： Tong, Shaocheng Sun, Kangkang Sui, Shuai Liaoning Univ Technol Coll Sci Jinzhou 121001 Peoples R China

In this paper, the problem of adaptive fuzzy decentralized optimal control is investigated for a class of nonlinear large-scale systems in strict-feedback form. The considered nonlinear large-scale systems contain the unknown nonlinear functions and unmeasured states. By utilizing the fuzzy logic systems to approximate the unknown nonlinear functions and cost functions, a fuzzy state observer is established to estimate the unmeasured states. The control design is divided into two phases. First, by using the state observer and the backstepping design technique, a feed-forward decentralized controller with parameters adaptive laws is designed, by which the original controlled strict-feedback nonlinear large-scale system is transformed into an equivalent affine nonlinear large-scale system. Second, by using adaptive dynamic programming theory, a feedback decentralized optimal controller is developed for the equivalent affine nonlinear system. The whole adaptive fuzzy decentralized optimal control scheme consists of a feed-forward decentralized controller and a feedback decentralized optimal controller. It is shown that the proposed adaptive fuzzy decentralized optimal control approach can guarantee that all the signals in the closed-loop system are bounded, and the tracking errors converge to a small neighborhood of zero. In addition, the proposed control approach can guarantee that the cost functions are minimized. Simulation results are given to demonstrate the effectiveness of the proposed control approach.

关键词： adaptive dynamic programming adaptive fuzzy control backstepping design technique decentralized optimal control strict-feedback nonlinear large-scale systems

来源：评论

学校读者我要写书评

暂无评论

Discrete-Time Multi-Player Games Based on Off-Policy Q-Learning

引用

IEEE ACCESS 2019年 7卷 134647-134659页

作者： Li, Jinna Xiao, Zhenfei Li, Ping Liaoning Shihua Univ Sch Informat & Control Engn Fushun 113001 Peoples R China Northeastern Univ State Key Lab Synthet Automat Proc Ind Shenyang 110819 Liaoning Peoples R China

In this paper, an off-policy game Q-learning algorithm is proposed for solving linear discrete-time non-zero sum multi-player game problems. Unlike the existing Q-learning methods for solving the Riccati equation by on-policy learning approaches for multi-player games, an off-policy game Q-learning method is developed for achieving the Nash equilibrium of multiple players. To this end, first, a non-zero sum game problem is formulated, and the value function and the Q-function defined according to each-player individual performance index are rigorously proved to be linear quadratic forms. Then, based on the dynamic programming and Q-learning methods, an off-policy game Q-learning algorithm is developed to find the control policies for multi-player games, such that the Nash equilibrium is reached under the learned control policies. The merit of this paper lies in that the proposed algorithm does not require the system model parameters to be known a priori and fully utilizes measurable data to learn the Nash equilibrium solution. Moreover, there is no bias of Nash equilibrium solution when implementing the proposed off-policy game Q-learning algorithm even though probing noises are added to control policies for maintaining the persistent excitation condition. While bias of the Nash equilibrium solution could be produced if on-policy game Q-learning is employed. This is another contribution of this paper

关键词： adaptive dynamic programming off-policy Q-learning non-zero sum game Nash equilibrium discrete-time systems

来源：评论

学校读者我要写书评

暂无评论

Multi-step heuristic dynamic programming for optimal control of nonlinear discrete-time systems

引用

INFORMATION SCIENCES 2017年 411卷 66-83页

作者： Luo, Biao Liu, Derong Huang, Tingwen Yang, Xiong Ma, Hongwen Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Guangdong Univ Technol Sch Automat Guangzhou 510006 Guangdong Peoples R China Tianjin Univ Sch Elect & Informat Engn Tianjin 300072 Peoples R China Texas A&M Univ Qatar POB 23874 Doha Qatar

Policy iteration and value iteration are two main iterative adaptive dynamic programming frameworks for solving optimal control problems. Policy iteration converges fast while requiring an initial stabilizing control policy, which is a strict constraint in practice. Value iteration avoids the requirement of initial admissible control policy while converging much slowly. This paper tries to utilize the advantages of policy iteration and value iteration, and avoids their drawbacks at the same time. Therefore, a multi-step heuristic dynamic programming (MsHDP) method is developed for solving the optimal control problem of nonlinear discrete-time systems. MsHDP speeds up value iteration and avoids the requirement of initial admissible control policy in policy iteration at the same time. The convergence theory of MsHDP is established by proving that it converges to the solution of the Bellman equation. For implementation purpose, the actor-critic neural network (NN) structure is developed. The critic NN is employed to estimate the value function and its NN weight vector is computed with a least-square scheme. The actor NN is used to estimate the control policy and a gradient descent method is proposed for updating its NN weight vector. According to the comparative simulation studies on two examples, the effectiveness and advantages of MsHDP are verified. (C) 2017 Elsevier Inc. All rights reserved.

关键词： Optimal control Multi-step heuristic dynamic programming adaptive dynamic programming Nonlinear systems Discrete-time Neural networks

来源：评论

学校读者我要写书评

暂无评论

Iterative ADP learning algorithms for discrete-time multi-player games

引用

ARTIFICIAL INTELLIGENCE REVIEW 2018年第1期50卷 75-91页

作者： Jiang, He Zhang, Huaguang Northeastern Univ Coll Informat Sci & Engn Shenyang 110819 Liaoning Peoples R China

adaptive dynamic programming (ADP) is an important branch of reinforcement learning to solve various optimal control issues. Most practical nonlinear systems are controlled by more than one controller. Each controller is a player, and to make a tradeoff between cooperation and conflict of these players can be viewed as a game. Multi-player games are divided into two main categories: zero-sum game and non-zero-sum game. To obtain the optimal control policy for each player, one needs to solve Hamilton-Jacobi-Isaacs equations for zero-sum games and a set of coupled Hamilton-Jacobi equations for non-zero-sum games. Unfortunately, these equations are generally difficult or even impossible to be solved analytically. To overcome this bottleneck, two ADP methods, including a modified gradient-descent-based online algorithm and a novel iterative offline learning approach, are proposed in this paper. Furthermore, to implement the proposed methods, we employ single-network structure, which obviously reduces computation burden compared with traditional multiple-network architecture. Simulation results demonstrate the effectiveness of our schemes.

关键词： adaptive dynamic programming Approximate dynamic programming Reinforcement learning Neural network

来源：评论

学校读者我要写书评

暂无评论

Intelligent Optimal Control With Critic Learning for a Nonlinear Overhead Crane System

引用

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 2018年第7期14卷 2932-2940页

作者： Wang, Ding He, Haibo Liu, Derong Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Chinese Acad Sci Inst Automat Beijing Engn Res Ctr Intelligent Syst & Technol Beijing 100190 Peoples R China Univ Chinese Acad Sci Sch Comp & Control Engn Beijing 100049 Peoples R China Univ Rhode Isl Dept Elect Comp & Biomed Engn Kingston RI 02881 USA Guangdong Univ Technol Sch Automat Guangzhou 510006 Guangdong Peoples R China

In this paper, for achieving the discounted optimal feedback stabilization of a nonlinear overhead crane system, we establish an intelligent control strategy to obtain the solution of the corresponding Hamilton-Jacobi- Bellman equation. Specifically, neural networks are employed to serve as a necessary component to the control system, which exhibits strong online learning ability. A novel updating rule compared to the traditional adaptive critic algorithms is developed, which eliminates the requirement of the initial stabilizing controller and brings in unique advantages to the adaptive critic control design. Stability analysis of the closed-loop system based on the well-known Lyapunov approach and experimental simulation considering the nonlinear overhead dynamics with different case studies are performed to verify the effectiveness of the present control method both in theory and applications.

关键词： adaptive dynamic programming artificial neural networks discounted optimal control intelligent control overhead crane

来源：评论

学校读者我要写书评

暂无评论

Manifold Regularized Reinforcement Learning

引用

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2018年第4期29卷 932-943页

作者： Li, Hongliang Liu, Derong Wang, Ding Tencent Inc AI Platform Dept Shenzhen 518057 Peoples R China Univ Sci & Technol Beijing Sch Automat & Elect Engn Beijing 100083 Peoples R China Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China

This paper introduces a novel manifold regularized reinforcement learning scheme for continuous Markov decision processes. Smooth feature representations for value function approximation can be automatically learned using the unsupervised manifold regularization method. The learned features are data-driven, and can be adapted to the geometry of the state space. Furthermore, the scheme provides a direct basis representation extension for novel samples during policy learning and control. The performance of the proposed scheme is evaluated on two benchmark control tasks, i.e., the inverted pendulum and the energy storage problem. Simulation results illustrate the concepts of the proposed scheme and show that it can obtain excellent performance.

关键词： adaptive dynamic programming approximate dynamic programming approximate policy iteration (API) manifold regularization reinforcement learning (RL)

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：