检索结果-内蒙古大学图书馆

Data-driven adaptive dynamic programming for partially observable nonzero-sum games via Q-learning method

INTERNATIONAL JOURNAL OF SYSTEMS SCIENCE 2019年第7期50卷 1338-1352页

作者： Wang, Wei Chen, Xin Fu, Hao Wu, Min China Univ Geosci Sch Automat Wuhan 430074 Hubei Peoples R China Hubei Key Lab Adv Control & Intelligent Automat C Wuhan 430074 Hubei Peoples R China

This paper concerns with a class of discrete-time linear nonzero-sum games with the partially observable system state. As is known, the optimal control policy for the nonzero-sum games relies on the full state measurement which is hard to fulfil in partially observable environment. Moreover, to achieve the optimal control, one needs to know the accurate system model. To overcome these deficiencies, this paper develops a data-driven adaptive dynamic programming method via Q-learning method using measurable input/output data without any system knowledge. First, the representation of the unmeasurable inner system state is built using historical input/output data. Then, based on the representation state, a Q-function-based policy iteration approach with convergence analysis is introduced to approximate the optimal control policy iteratively. A neural network (NN)-based actor-critic framework is applied to implement the developed data-driven approach. Finally, two simulation examples are provided to demonstrate the effectiveness of the developed approach.

关键词： adaptive dynamic programming nonzero-sum games partially observable Q-learning

来源：评论

学校读者我要写书评

暂无评论

Modified general policy iteration based adaptive dynamic programming for unknown discrete-time linear systems

引用

INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL 2022年第12期32卷 7149-7173页

作者： Jiang, Huaiyuan Zhou, Bin Duan, Guang-Ren Harbin Inst Technol Ctr Control Theory & Guidance Technol Harbin 150001 Peoples R China

In this article, the general policy iteration (GPI) method for the optimal control of discrete-time linear systems is studied. First, the existing result on the GPI method is recalled and some new properties are proposed. Based on these new properties, a model-based modified GPI algorithm is proposed with its convergence proof. Moreover, the data-driven implementation for the proposed method is introduced without using the information of the system matrices. Compared with the existing results, the condition to initiate the GPI based algorithm is further relaxed by the proposed algorithm. The simulation example verifies the effectiveness of the proposed modified GPI based algorithm.

关键词： adaptive dynamic programming data-driven control discrete-time systems general policy iteration policy iteration unknown systems

来源：评论

学校读者我要写书评

暂无评论

Policy gradient adaptive dynamic programming for nonlinear discrete-time zero-sum games with unknown dynamics

引用

SOFT COMPUTING 2023年第9期27卷 5781-5795页

作者： Lin, Mingduo Zhao, Bo Liu, Derong Beijing Normal Univ Sch Syst Sci Beijing 100875 Peoples R China Chongqing Univ Posts & Telecommun Key Lab Ind Internet Things & Networked Control Minist Educ Chongqing 400065 Peoples R China Southern Univ Sci & Technol Dept Mech & Energy Engn Shenzhen 518055 Peoples R China Univ Illinois Dept Elect & Comp Engn Chicago IL 60607 USA

A novel policy gradient (PG) adaptive dynamic programming method is developed to deal with nonlinear discrete-time zero-sum games with unknown dynamics. To facilitate the implementation, a policy iteration algorithm is established to approximate the iterative Q-function, as well as the control and disturbance policies via three neural network (NN) approximators, respectively. Then, the iterative Q-function is exploited to update the control and disturbance policies via PG method. To stabilize the training process and improve the data usage efficiency, the experience replay technique is applied to train the weight vectors of the three NNs by using mini-batch empirical data from replay memory. Furthermore, the convergence in terms of the iterative Q-function is proved. Simulation results of two numerical examples are provided to show the effectiveness of the proposed method.

关键词： adaptive dynamic programming Reinforcement learning Neural networks Zero-sum games Policy gradient

来源：评论

学校读者我要写书评

暂无评论

Event-Triggered Model Predictive adaptive dynamic programming for Road Intersection Path Planning of Unmanned Ground Vehicle

引用

IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY 2021年第11期70卷 11228-11243页

作者： Hu, Chaofang Zhao, Lingxue Qu, Ge Tianjin Univ Sch Elect & Informat Engn Tianjin 300072 Peoples R China

Autonomous driving of unmanned ground vehicle (UGV) at road intersection is a challenging task due to the complicated traffic conditions. In this paper, an event-triggered model predictive adaptive dynamic programming (MPADP) algorithm is proposed for path planning of UGV at road intersection. Following the critic-actor scheme of adaptive dynamic programming (ADP), cost function approximation and control policy generation are combined to formulate MPADP. The infinite horizon cost function of ADP is stacked over predictive horizon of model predictive control (MPC), and then the infinite horizon cost function is converted to the finite horizon-stacked cost function in MPADP. By minimizing the approximation error within predictive horizon, the approximation accuracy is enhanced. Considering the limitation of energy consumption, the event-triggered mechanism is designed based on the mismatch of cost function approximation. Three triggering conditions are designed, and the corresponding boundedness of approximation error is proved. Simulation results illustrate the effectiveness, efficiency and feasibility in application of the event-triggered MPADP method for path planning at road intersection.

关键词： Roads Path planning Cost function dynamic programming Safety Predictive models Kinematics Unmanned ground vehicle path planning event-triggered adaptive dynamic programming model predictive control

来源：评论

学校读者我要写书评

暂无评论

An adaptive dynamic programming method for observer-based sliding mode control of connected vehicles subject to deception attacks

引用

INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL 2024年第6期34卷 3659-3678页

作者： Xu, Yangguang Guo, Ge Yu, Shuanghe Dalian Maritime Univ Marine Elect Engn Coll Dalian Peoples R China Northeastern Univ State Key Lab Synthet Automat Proc Ind Shenyang Peoples R China Northeastern Univ Qinhuangdao Sch Control Engn Qinhuangdao Peoples R China

This article investigates the problem of optimal observer-based sliding mode control (SMC) of connected vehicles subject to deception attacks and disturbances with adaptive dynamic programming (ADP) method. For a group of vehicles with unknown nonlinear dynamics term and disturbance, this article aims to give a control methodology to achieve secure tracking of the desired spacing, velocity and acceleration. A neural network (NN) and an observer are constructed to estimate the unknown nonlinear term and the states, respectively. Then, a SMC scheme incorporating NN approximation is developed and an off-policy ADP method is used to implement the optimal control of sliding mode dynamics. The proposed method can ensure individual stability and string stability of the set of vehicles. Numerical simulations are conducted to demonstrate the validity of the proposed controller.

关键词： adaptive dynamic programming connected vehicles cyber-attacks neural network sliding mode control

来源：评论

学校读者我要写书评

暂无评论

adaptive Learning and Control for MIMO System Based on adaptive dynamic programming

引用

IEEE TRANSACTIONS ON NEURAL NETWORKS 2011年第7期22卷 1133-1148页

作者： Fu, Jian He, Haibo Zhou, Xinmin Wuhan Univ Technol Sch Automat Wuhan 430070 Hubei Peoples R China

adaptive dynamic programming (ADP) is a promising research field for design of intelligent controllers, which can both learn on-the-fly and exhibit optimal behavior. Over the past decades, several generations of ADP design have been proposed in the literature, which have demonstrated many successful applications in various benchmarks and industrial applications. While many of the existing researches focus on multiple-inputs-single-output system with steepest descent search, in this paper we investigate a generalized multiple-input-multiple-output (GMIMO) ADP design for online learning and control, which is more applicable to a wide range of practical real-world applications. Furthermore, an improved weight-updating algorithm based on recursive Levenberg-Marquardt methods is presented and embodied in the GMIMO approach to improve its performance. Finally, we test the performance of this approach based on a practical complex system, namely, the learning and control of the tension and height of the looper system in a hot strip mill. Experimental results demonstrate that the proposed approach can achieve effective and robust performance.

关键词： adaptive dynamic programming Levenberg-Marquardt method looper system multiple-input-multiple-output online learning and control

来源：评论

学校读者我要写书评

暂无评论

Distributed cooperative H_∞ optimal tracking control of MIMO nonlinear multi-agent systems in strict-feedback form via adaptive dynamic programming

引用

INTERNATIONAL JOURNAL OF CONTROL 2018年第4期91卷 952-968页

作者： Luy, N. T. Ind Univ Ho Chi Minh City Elect Technol Dept Ho Chi Minh City Vietnam

The design of distributed cooperative H-infinity optimal controllers for multi-agent systems is a major challenge when the agents' models are uncertain multi-input and multi-output nonlinear systems in strict-feedback form in the presence of external disturbances. In this paper, first, the distributed cooperative H-infinity optimal tracking problem is transformed into controlling the cooperative tracking error dynamics in affine form. Second, control schemes and online algorithms are proposed via adaptive dynamic programming (ADP) and the theory of zero-sum differential graphical games. The schemes use only one neural network (NN) for each agent instead of three from ADP to reduce computational complexity as well as avoid choosing initial NN weights for stabilising controllers. It is shown that despite not using knowledge of cooperative internal dynamics, the proposed algorithms not only approximate values to Nash equilibrium but also guarantee all signals, such as the NN weight approximation errors and the cooperative tracking errors in the closed-loop system, to be uniformly ultimately bounded. Finally, the effectiveness of the proposed method is shown by simulation results of an application to wheeled mobile multi-robot systems.

关键词： Nonlinear systems in strict-feedback form cooperative H-infinity optimal control adaptive dynamic programming zero-sum differential graphical game

来源：评论

学校读者我要写书评

暂无评论

Optimal synchronization with L ²-gain performance: An adaptive dynamic programming approach

引用

NEURAL NETWORKS 2024年 179卷 106566页

作者： Chen, Zitao Chen, Kairui Tang, Ruizhi Guangzhou Univ Sch Mech & Elect Engn Guangzhou 510006 Guangdong Peoples R China Southeast Univ Sch Math Nanjing 211189 Peoples R China Seoul Natl Univ Dept Math Sci Seoul 08826 South Korea

This paper studies an optimal synchronous control protocol design for nonlinear multi-agent systems under partially known dynamics and uncertain external disturbance. Under some mild assumptions, Hamilton-Jacobi- Isaacs equation is derived by the performance index function and system dynamics, which serves as an equivalent formulation. Distributed policy iteration adaptive dynamic programming is developed to obtain the numerical solution to the Hamilton-Jacobi-Isaacs equation. Three theoretical results are given about the proposed algorithm. First, the iterative variables is proved to converge to the solution to Hamilton-Jacobi- Isaacs equation. Second, the L 2-gain performance of the closed loop system is achieved. As a special case, the origin of the nominal system is asymptotically stable. Third, the obtained control protocol constitutes an Nash equilibrium solution. Neural network-based implementation is designed following the main results. Finally, two numerical examples are provided to verify the effectiveness of the proposed method.

关键词： Multi-agent system L2-gain performance adaptive dynamic programming Nash equilibrium Neural network

来源：评论

学校读者我要写书评

暂无评论

Event-Triggered adaptive dynamic programming for Hierarchical Sliding-Mode Surface-Based Optimal Control of Switched Nonlinear Systems

引用

IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING 2024年第3期21卷 4851-4863页

作者： Zhang, Haoyan Zhang, Yingwei Zhao, Xudong Northeastern Univ Sch Informat Sci & Engn Shenyang 110819 Peoples R China Northeastern Univ State Key Lab Synth Automat Proc Ind Shenyang 110819 Peoples R China Dalian Univ Technol Fac Elect Informat & Elect Engn Dalian 116024 Peoples R China Bohai Univ Coll Control Sci & Engn Jinzhou 121013 Peoples R China

In this paper, a novel hierarchical sliding-mode surface (HSMS)-based adaptive event-triggered optimal control (ETOC) method is proposed for switched nonlinear systems using single-critic adaptive dynamic programming (ADP). Initially, an ameliorative HSMS containing all system states is constructed to improve the system response rate. With positive quadratic functions being assigned for the developed HSMS-based cost function, the original stabilization problem is further transformed into an array of ETOC problems. Subsequently, by implementing a single-critic neural network (NN) whose weights are tuned based on an aperiodic manner and the historical data, a series of event-triggered Hamilton-Jacobi-Bellman (HJB) equations are approximately solved. Meanwhile, the condition for the persistence of excitation existing in the traditional actor-critic dual NNs framework is relaxed. Furthermore, the uniformly ultimate boundedness (UUB) of all signals in the closed-loop system is guaranteed by the Lyapunov extension theorem. Finally, a practical continuous stirred tank reactor plant is provided to substantiate the efficacy of the proposed HSMS-based adaptive single-critic ETOC method. Note to Practitioners-Switched nonlinear systems have attracted much attention due to their flexibility in characterizing multi-mode features in practice. The control synthesis and stability analysis of switched nonlinear systems have been the research focus in the control field. However, with the development of adaptive dynamic programming (ADP), it is no longer limited to the study of control synthesis and stability of switched systems. How to improve the response rate and achieve optimal control of switched nonlinear systems with less control cost becomes critical. On the other hand, the persistent information transmission of control signals not only occupies more bandwidth, but also causes severe transmission delays, thus it is important to conduct the signal transmission only when necessa

关键词： adaptive dynamic programming single-critic neural network event-triggered control switched nonlinear systems hierarchical sliding mode surface

来源：评论

学校读者我要写书评

暂无评论

Design and implementation of an optimal switching controller for uninterruptible power supply inverters using adaptive dynamic programming

引用

IET POWER ELECTRONICS 2019年第12期12卷 3068-3076页

作者： Gogani Khiabani, Ataollah Heydari, Ali Southern Methodist Univ Dept Mech Engn 6425 Boaz Lane Dallas TX 75205 USA

In this study, a new approach based on adaptive dynamic programming (ADP) is proposed to control single-phase uninterruptible power supply inverters. The control scheme uses a single function approximator, called critic, to evaluate the optimal cost and determine the optimal switching. After offline training of the critic, which is a function of system states and elapsed time, the resulting optimal weights are used in online control, to get a smooth output AC voltage in a feedback form. Simulations show the desirable performance of this controller with linear and non-linear loads and its relative robustness to parameter uncertainty and disturbances. Furthermore, the proposed controller is upgraded so that the inverter is suitable for single-phase variable frequency drives. Finally, as one of the few studies in the field of ADP, the proposed controllers are implemented on a physical prototype to show the performance in practise.

关键词： dynamic programming invertors variable speed drives function approximation uninterruptible power supplies feedback optimal switching controller ADP adaptive dynamic programming single-phase uninterruptible power supply inverters control scheme single function approximator called critic optimal cost offline training system states elapsed time resulting optimal weights online control smooth output AC voltage nonlinear loads inverter single-phase variable frequency drives

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：