检索结果-内蒙古大学图书馆

Off-policy learning for adaptive optimal output synchronization of heterogeneous multi-agent systems

AUTOMATICA 2020年 119卷 109081-109081页

作者： Chen, Ci Lewis, Frank L. Xie, Kan Xie, Shengli Liu, Yilu Guangdong Univ Technol Sch Automat Guangdong Key Lab IoT Informat Technol Guangzhou Peoples R China Nanyang Technol Univ Sch Elect & Elect Engn Singapore Singapore Univ Texas Arlington UTA Res Inst Ft Worth TX USA Minist Educ Joint Int Res Lab Intelligent Informat Proc & Sys Guangzhou Peoples R China Guangdong HongKong Macao Joint Lab Smart Discrete Guangzhou Peoples R China Univ Tennessee Dept Elect Engn & Comp Sci Knoxville TN USA Oak Ridge Natl Lab Oak Ridge TN USA

This paper proposes an off-policy learning-based dynamic state feedback protocol that achieves the optimal synchronization of heterogeneous multi-agent systems (MAS) over a directed communication network. Note that most of the recent works on heterogeneous MAS are not formed in an optimal manner. By formulating the cooperative output regulation problem as an H-infinity optimization problem, we can use reinforcement learning to find output synchronization protocols online along with the system trajectories without solving output regulator equations. In contrast to the existing optimal literature where leader's states are assumed to be globally or distributively available for the communication, we only allow the relative system outputs to transmit through the network;namely, no leader's states are needed now for the control or learning purpose. (C) 2020 Elsevier Ltd. All rights reserved.

关键词： Off-policy learning adaptive dynamic programming Reinforcement learning Optimal synchronization Heterogeneous multi-agent system

来源：评论

学校读者我要写书评

暂无评论

Output Feedback H∞ Control for Linear Discrete-Time Multi-Player Systems With Multi-Source Disturbances Using Off-Policy Q-Learning

引用

IEEE ACCESS 2020年 8卷 208938-208951页

作者： Xiao, Zhenfei Li, Jinna Li, Ping Liaoning Shihua Univ Sch Informat & Control Engn Fushun 113001 Liaoning Peoples R China

In this paper, a data-driven optimal control method based on adaptive dynamic programming and game theory is presented for solving the output feedback solutions of the H-infinity control problem for linear discrete-time systems with multiple players subject to multi-source disturbances. We first transform the H-infinity control problem into a multi-player game problem following the theoretical solutions according to game theory. Since the system state may not be measurable, we derive the output feedback based control policies and disturbances through mathematical operations. Considering the advantages of off-policy reinforcement learning (RL) over on-policy RL, a novel off-policy game Q-learning algorithm dealing with mixed competition and cooperation among players is developed, such that the H-infinity control problem can be finally solved for linear multi-player systems without the knowledge of system dynamics. Moreover, rigorous proofs of algorithm convergence and unbiasedness of solutions are presented. Finally, simulation results demonstrated the effectiveness of the proposed method.

关键词： Output feedback H-infinity control adaptive dynamic programming game theory reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Data-driven Structural Control of Monopile Wind Turbine Towers Based on Machine Learning ⁎

引用

IFAC-PapersOnLine 2020年第2期53卷 7466-7471页

作者： Jincheng Zhang Xiaowei Zhao Xing Wei School of Engineering University of Warwick Coventry UK

This paper studies the data-driven structural control of monopile wind turbine towers based on machine learning approach, by using an active tuned mass damper (TMD) located in the nacelle. The adaptive dynamic programming (ADP) approach is employed to obtain the optimal controller which is derived on the modern large-scale machine learning platform Tensorflow. The proposed network structure includes three simple three-layer neural networks (NNs), i.e. a plant network, a critic network, and an action network. The plant network is used to capture the fully nonlinear dynamics of the structural system while the action network is used to approximate the optimal controller. Their training requires the gradient information flowing through the whole network. The automatic differentiation is used in this paper for all the gradient derivations, which greatly improves the employed ADP algorithm’s ability in solving complex practical problems. The simulation results of structural control of monopile turbine towers show that on average the active TMD achieves 15% performance improvement on tower fatigue load reduction over a passive TMD, with small active power consumption (less than 0.24% of the turbine’s nominal power production). Besides, the controller design considers the trade-off between control performance and power consumption.

关键词： adaptive dynamic programming Reinforcement Learning Neural Networks Floating Wind Turbine Active Structural Control

来源：评论

学校读者我要写书评

暂无评论

Stackelberg Differential Game-based Optimal Fault Estimation and Accommodation for Continuous-time Linear Systems

引用

IFAC-PapersOnLine 2020年第2期53卷 13751-13756页

作者： Yuhang Xu Hao Yang Bin Jiang Marios M. Polycarpou The College of Automation Engineering Nanjing University of Aeronautics and Astronautics Nanjing 210016 China KIOS Research and Innovation Center of Excellence and the Dept. of Electrical and Computer Engineering University of Cyprus Nicosia 1678 Cyprus

This paper deals with the optimal fault estimation and accommodation problem for a class of linear systems in the framework of Stackelberg differential game theory. In this framework, the observer plays the role of the follower, while the system plays the role of the leader in making sequential decisions. A dual controller approach is used to design an auxiliary controller for the observer such that it can non-cooperate with the controller of the system to achieve the Stackelberg equilibrium. To achieve the online updating of the fault-tolerant controller, an adaptive dynamic programming methodology is used by establishing two critic neural networks for the observer and system respectively. Finally, a simulation is presented to illustrate the efficiency and applicability of the theoretical results.

关键词： Optimal fault estimation optimal fault-tolerant control Stackelberg differential game adaptive dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Observer-Based H ∞ Tracking Control Scheme and Its Application to Robot Arms ⁎

引用

IFAC-PapersOnLine 2020年第5期53卷 536-541页

作者： Shunchao Zhang Bo Zhao Derong Liu Yongwei Zhang School of Automation Guangdong University of Technology Guangzhou 510006 China School of Systems Science Beijing Normal University Beijing 100875 China

In this paper, an adaptive dynamic programming (ADP)-based H ∞ tracking control method is proposed for unknown continuous-time nonlinear systems. By constructing a neural network (NN) observer with system input and output data, the dynamics of completely unknown systems can be identified online. Then, an augmented system composed of tracking error and reference trajectory dynamics is formulated, and the H ∞ tracking control problem is regarded as a zero-sum game problem which is solved by using ADP technique. A critic NN is employed to approximate the cost function to solve the Hamilton-Jacobi-Isaacs equation. Furthermore, the ultimate uniform boundedness of observation errors, critic NN weights, and tracking errors are all proved through Lyapunov’s direct method. Finally, the proposed method is applied to a single link robot arm with unknown dynamics to demonstrate its effectiveness.

关键词： adaptive dynamic programming H ∞ tracking control zero-sum game neural networks robot arms

来源：评论

学校读者我要写书评

暂无评论

Approximate Optimal Sliding Mode Tracking Control for Modular Reconfigurable Robots Based on Critic-only Structure ⁎

引用

IFAC-PapersOnLine 2020年第5期53卷 554-559页

作者： Hongbing Xia Fuquan Xia Bo Zhao Yinghui Huang School of Electronic and Electrical Engineering Bengbu University Bengbu 233030 P.R. China School of System Science Beijing Normal University Beijing 100875 P.R. China School of Science Bengbu University Bengbu 233030 P.R. China

In this paper, an approximate optimal sliding mode tracking control (SMTC) strategy is investigated for modular reconfigurable robots (MRRs) through critic-only structure-based adaptive dynamic programming (ADP) scheme. The SMTC is achieved by three parts, i.e., the optimal control of the nominal system, the sliding mode-based iterative control and an adaptive robust term. The sliding mode-based iterative controller suppresses the error caused by the trajectory tracking, and the adaptive robust term is employed to ensure the reachable condition of sliding mode surface. By solving the Hamilton-Jacobi-Bellman equation with the critic neural network only, the sliding mode-based iterative control can be derived. The SMTC strategy can drive the MRR to present achieve a faster control action based on the approximate optimal control. The closed-loop MRR system is guaranteed to be asymptotically stable under the developed SMTC policy. At last, the effectiveness of the presented strategy was validated via the comparative simulation.

关键词： adaptive dynamic programming Sliding mode tracking control Neural networks Modular reconfigurable robots

来源：评论

学校读者我要写书评

暂无评论

A Combined Policy Gradient and Q-learning Method for Data-driven Optimal Control Problems 9

A Combined Policy Gradient and Q-learning Method for Data-dr...

引用

9th International Conference on Information Science and Technology (ICIST)

作者： Lin, Mingduo Liu, Derong Zhao, Bo Dai, Qionghai Dong, Yi Guangdong Univ Technol Sch Automat Guangzhou Peoples R China Beijing Normal Univ Sch Syst Sci Beijing Peoples R China Tsinghua Univ Dept Automat Beijing Peoples R China Beijing Inst Technol Sch Opt & Photon Beijing Peoples R China

ISBN: (纸本)9781728121062

This paper focuses on the data-driven controller design for optimal control problems of nonlinear nonaffine discrete-time systems. A novel policy gradient and Q-learning (PGQL) adaptive algorithm which learns the optimal control policy from real empirical data is developed without requiring system dynamics. A policy iteration scheme is designed to iteratively update the approximate Q-function, and the control policy is improved via gradient method until they converge to the bounded regions of the optimal Q-function and the optimal control policy, respectively. Two neural networks (NNs) are employed to realize the developed algorithm. Moreover, the convergence analysis of approximate Q-function is established. Since the control policy is parameterized, it can be upgraded through updating the actor-NN parameters in the direction of the performance gradient. Finally, the simulation results are given to verify the performance of the developed PGQL adaptive algorithm.

关键词： adaptive dynamic programming optimal control reinforcement learning policy gradient Q-learning data-driven

来源：评论

学校读者我要写书评

暂无评论

Neural Network Tracking Control of Unknown Servo System with Approximate dynamic programming

Neural Network Tracking Control of Unknown Servo System with...

引用

第三十八届中国控制会议

作者： Yongfeng Lv Xuemei Ren Tianyi Zeng Linwei Li Jing Na School of Automation Beijing Institute of Technology IEEE Faculty of Mechanical & Electrical Engineering Kunming University of Science & Technology

Although the adaptive dynamic programming(ADP) scheme has been widely researched on the optimal problem in recent years, which has not been applied to the servo system. In this paper, a simplified reinforcement learning(RL) based(ADP)scheme is developed to obtain the optimal tracking control of the servo system, where the unknown system dynamics are approximated with a three-layer neural network(NN) identifier. First, the servo system model is constructed and a three-layer NN identifier is used to approximate the unknown servo system. The NN weights of both the hidden layer and output layer are synchronously tuned with an adaptive gradient law. An RL-based critic NN is then used to learn the optimal cost function, and NN weights are updated by minimizing the squared Hamilton-Jacobi-Bellman(HJB) error. The optimal tracking control of the servomechanism is obtained based on the three-layer NN identifier and RL scheme, which can make the motor speed track the predefined command. Moreover, the convergence of the identifier and NN weights is proved. Finally, a servomechanism model is provided, which can illustrate the proposed methods.

关键词： Reinforcement Learning adaptive dynamic programming Optimal Control Neural Networks Servomechanisms

来源：评论

学校读者我要写书评

暂无评论

adaptive dynamic programming for Finite-Horizon Optimal Control of Discrete-Time Nonlinear Systems with ε-Error Bound

引用

IEEE TRANSACTIONS ON NEURAL NETWORKS 2011年第1期22卷 24-36页

作者： Wang, Fei-Yue Jin, Ning Liu, Derong Wei, Qinglai Chinese Acad Sci Inst Automat Key Lab Complex Syst & Intelligence Sci Beijing 100190 Peoples R China Univ Illinois Dept Elect & Comp Engn Chicago IL 60607 USA

In this paper, we study the finite-horizon optimal control problem for discrete-time nonlinear systems using the adaptive dynamic programming (ADP) approach. The idea is to use an iterative ADP algorithm to obtain the optimal control law which makes the performance index function close to the greatest lower bound of all performance indices within an epsilon-error bound. The optimal number of control steps can also be obtained by the proposed ADP algorithms. A convergence analysis of the proposed ADP algorithms in terms of performance index function and control policy is made. In order to facilitate the implementation of the iterative ADP algorithms, neural networks are used for approximating the performance index function, computing the optimal control policy, and modeling the nonlinear system. Finally, two simulation examples are employed to illustrate the applicability of the proposed method.

关键词： adaptive critic designs adaptive dynamic programming approximate dynamic programming learning control neural control neural dynamic programming optimal control reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Research on Optimal Electrical Energy Management Method of Smart Grid Based on Value Iteration

引用

IFAC-PapersOnLine 2020年第5期53卷 681-685页

作者： Chunsheng Yan Yongjian Zheng Xun Dai Baiyu Gao Liguang Zhong Shi Xing Ruizhuo Song Liaoning Electric Power Development Stock Co. LtdShenyang 110179China State Grid Liaoning Electric Power Supply Co. LtdShenyang 110006China School of Automation and Electrical Engineering University of Science and Technology Beijing Beijing 100083 China

Based on the smart grid residential load data and energy storage terminal data, this paper expands a new optimal control method of smart grid supply and demand matching and *** paper describes the operation principle and defines the required optimization parameters of the control object—electric energy storage management system. Aiming at the characteristics of discrete non linearity of the smart grid, a value iteration algorithm is selected to establish a control strategy,and an adaptive value iteration optimization scheme is *** use the BP neural network to export the control strategy of the electric energy management system by reducing the back propagation error. Finally,we use the day-ahead electricity price data to achieve numerical *** result shows that the adaptive value iteration algorithm methods have higher efficiency and superior adaptability compared with the traditional BP neural network control method.

关键词： adaptive dynamic programming optimal control value iteration smart grid BP neural network

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：