检索结果-内蒙古大学图书馆

A Scalable Low-Power Reconfigurable Accelerator for Action-Dependent Heuristic dynamic programming

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS I-REGULAR PAPERS 2018年第6期65卷 1897-1908页

作者： Zheng, Nan Mazumder, Pinaki Univ Michigan Dept Elect Engn Comp Sci Ann Arbor MI 48104 USA

adaptive dynamic programming (ADP) is an effective algorithm that has been successfully deployed in various control tasks. For many emerging applications where power consumption is a major design consideration, the conventional way of implementing ADP as software executing on a general-purpose processor is not sufficient. This paper proposes a scalable and low-power hardware architecture for implementing one of the most popular forms of ADP called action-dependent heuristic dynamic programming. Different from most machine-learning accelerators that mainly focus on the inference operation, the proposed architecture is also designed for energy-efficient learning, considering the highly iterative and interactive nature of the ADP algorithm. In addition, a virtual update technique is proposed to speed up the computation and to improve the energy efficiency of the accelerators. Two design examples are presented to demonstrate the proposed algorithm and architecture. Compared with the software approach running on a general-purpose processor, the accelerator operating at 175 MHz achieves 270 times improvement in computational time while consuming merely 25 mW power. Furthermore, it is demonstrated that the proposed virtual update algorithm can effectively boost the energy efficiency of the accelerator. Improvements up to 1.64 times are observed in the benchmark tasks employed.

关键词： adaptive dynamic programming neural networks low-power accelerators action-dependent heuristic dynamic programming machine learning

来源：评论

学校读者我要写书评

暂无评论

A novel triggering condition of event-triggered control based on heuristic dynamic programming for discrete-time systems

引用

OPTIMAL CONTROL APPLICATIONS & METHODS 2018年第4期39卷 1467-1478页

作者： Wang, Ziyang Wei, Qinglai Liu, Derong Univ Sci & Technol Beijing Sch Automat & Elect Engn Beijing Peoples R China Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Guangdong Univ Technol Sch Automat Guangzhou Guangdong Peoples R China

In this paper, an event-triggered heuristic dynamic programming algorithm for discrete-time nonlinear systems with a novel triggering condition is studied. Different from traditional heuristic dynamic programming algorithms, the control law in this algorithm will only be updated when the triggering condition is satisfied to reduce the computational burden. Three neural networks are employed, which are model network, action network, and critic network. Model functions, control laws, and value functions are estimated using neural networks, respectively. The main contribution of this algorithm is the novel triggering condition with simpler form and fewer assumptions. Additionally, a proof of the stability for discrete-time systems using Lyapunov technique is given. Finally, two simulations are shown to verify the effectiveness of the developed algorithm.

关键词： adaptive dynamic design adaptive dynamic programming approximate dynamic programming event-triggered control optimal control

来源：评论

学校读者我要写书评

暂无评论

Integral reinforcement learning off-policy method for solving nonlinear multi-player nonzero-sum games with saturated actuator

引用

NEUROCOMPUTING 2019年 335卷 96-104页

作者： Ren, He Zhang, Huaguang Wen, Yinlei Liu, Chong Northeastern Univ Coll Informat Sci & Engn Box 134 Shenyang 110819 Liaoning Peoples R China Northeastern Univ Key Lab Integrated Automat Proc Ind Natl Educ Minist Shenyang 110004 Liaoning Peoples R China

In this paper, an effective off-policy algorithm is proposed to solve the continuous time nonzero-sum (NZS) control problem for unknown nonlinear systems with saturated actuator. A class of nonquadratic function is used to construct the performance functions to deal with constrained inputs. Utilizing the integral reinforcement learning (IRL) technique, the off-policy learning mechanism is introduced to design an iterative method for the continuous-time NZS constrained control problem without requiring the knowledge of system dynamics. To show the convergence of the proposed method, the traditional policy iteration (PI) method is discussed for the continuous-time NZS control problem with saturated actuator at first. Then, the equivalence of the proposed method with the traditional PI method is proved. Neural networks are introduced to construct the actor-critic structure, where the critic neural networks are aimed at approximating the iterative value functions and the actor neural networks are aimed at approximating the iterative control policies. Finally, two cases are simulated to verify the effectiveness of the proposed method. (C) 2019 Published by Elsevier B.V.

关键词： adaptive dynamic programming Nonzero-sum game Reinforcement learning Off-policy Constrained optimal control

来源：评论

学校读者我要写书评

暂无评论

Event-triggered optimal consensus tracking control for multi-agent systems with unknown internal states and disturbances

引用

NONLINEAR ANALYSIS-HYBRID SYSTEMS 2019年 33卷 227-248页

作者： Zhao, Wei Yu, Wenwu Zhang, Huaipin Southeast Univ Sch Math Jiangsu Prov Key Lab Networked Collect Intelligen Nanjing 210096 Jiangsu Peoples R China Nanjing Univ Posts & Telecommun Inst Adv Technol Nanjing 210023 Jiangsu Peoples R China

In this paper, we propose an event-triggered consensus tracking control for unknown nonlinear multi-agent systems (NMASs) with disturbances by using adaptive dynamic programming (ADP) technique. Considering the disturbances as control inputs, the optimal consensus tracking control problem can be transformed into multi-player zero-sum differential games, where the controllers are designed to minimize the cost functions under the worst-case disturbances. To recover the unknown internal states and control coefficient functions, neural network (NN)-based observers are established. Then, NN-based critics are applied to approximate the value functions and help calculate the optimal control policies and disturbance policies. In order to save the computation resource and reduce the transmission load, the designed observers and controllers are updated only when the designed events are triggered. Stability of the proposed method is demonstrated by Lyapunov analysis for both the continuous and jump dynamics. Meanwhile, we can obtain that the weight estimated errors of the observer NNs, the critic NNs and the local neighbor consensus tracking errors are uniformly ultimately bounded (UUB). Finally, a simulation example is given to illustrate the effectiveness of the proposed approach. (c) 2019 Elsevier Ltd. All rights reserved.

关键词： Multi-agent systems adaptive dynamic programming Event-triggered control NN-based observer Zero-sum differential games

来源：评论

学校读者我要写书评

暂无评论

adaptive Optimal Control With Guaranteed Convergence Rate for Continuous-Time Line it Systems With Completely Unknown dynamics

引用

IEEE ACCESS 2019年 7卷 11526-11532页

作者： Zhang, Kai Ge, Suo-Liang Hefei Univ Technol Hefei 230009 Anhui Peoples R China

The traditional linear quadratic optimal control can be summarized as finding the state feedback controller so that the closed-loop system is stable and the performance index is minimum. And, it is well-known that the solution of the linear quadratic optimal control problem can be solved by algebraic Riccati equation. However, the feature of the traditional linear quadratic optimal control theory is that the convergence rate is not specified. This may result in the phenomena of slow convergence. In this paper, we mainly consider the linear quadratic optimal control with guaranteed convergence rate (LQOCGCR) and propose a policy iteration-based adaptive dynamic programming algorithm that includes offline and online versions for finding the solution of the LQOCGCR. Finally, a numerical example is worked out to show the effectiveness of the proposed approach.

关键词： Linear quadratic optimal control convergence rate policy iteration adaptive dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Synchronous optimal control method for nonlinear systems with saturating actuators and unknown dynamics using off-policy integral reinforcement learning

引用

NEUROCOMPUTING 2019年 356卷 162-169页

作者： Zhang, Zenglian Song, Ruizhuo Cao, Min Univ Sci & Technol Beijing Donlinks Sch Econ & Management Beijing 100083 Peoples R China Univ Sci & Technol Beijing Sch Automat & Elect Engn Beijing 100083 Peoples R China China Energy Engn Investment Corp Ltd Beijing 100022 Peoples R China

The present study establishes an approximate optimal critic learning algorithm, based on the single-network integral reinforcement learning (IRL) algorithm and intends to solve the optimal control problem for an unknown nonlinear system with saturating actuators. The value function is formulated through building generalized nonquadratic functions. In order to solve the Hamilton-Jacobi-Bellman (HJB) equation, a novel optimal scheme for the control approximation, based on the off-policy iteration is presented. Moreover, the single-neural network implementation procedure is introduced to complete the iteration algorithm. The synchronous IRL policy iteration is proposed to update the weight of the critic neural network. Finally, reasonable simulation results are provided for confirming the effectiveness of the proposed optimal approximation control technique in solving equations for a linear and oscillating systems. (C) 2019 Elsevier B.V. All rights reserved.

关键词： Approximate dynamic programming adaptive dynamic programming Off-policy Integral reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Generalized Policy Iteration-based Reinforcement Learning Algorithm for Optimal Control of Unknown Discrete-time Systems

Generalized Policy Iteration-based Reinforcement Learning Al...

引用

第33届中国控制与决策会议

作者： Mingduo Lin Bo Zhao Derong Liu Xi Liu Fangchao Luo School of Automation Guangdong University of Technology School of Systems Science Beijing Normal University

This paper presents a novel generalized policy iteration-based reinforcement learning(RL) algorithm to deal with infinite-horizon optimal control problems of nonlinear discrete-time systems with completely unknown *** the present iterative algorithm,two iteration procedures are utilized to obtain the iterative Q-function and the iterative control ***,the iterative Q-function is obtained by the temporal difference learning and the policy gradient method is utilized to directly optimize the iterative control ***,the convergence and optimality analysis of the generalized policy iteration-based RL algorithm are *** implement this algorithm,two neural networks,including a critic network and an action network,are used to approximate the iterative Q-function and the iterative control ***,a numerical simulation example is provided to illustrate the effectiveness of the proposed control method.

关键词： adaptive dynamic programming Reinforcement learning Generalized policy iteration Neural networks Optimal control Unknown Discrete-time Systems

来源：评论

学校读者我要写书评

暂无评论

Decentralized robust zero-sum neuro-optimal control for modular robot manipulators in contact with uncertain environments: theory and experimental verification

引用

NONLINEAR dynamicS 2019年第1期97卷 503-524页

作者： Dong, Bo An, Tianjiao Zhou, Fan Liu, Keping Li, Yuanchun Changchun Univ Technol Dept Control Sci & Engn Changchun 130012 Jilin Peoples R China Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China

This paper presents a decentralized robust zero-sum optimal control approach for modular robot manipulators (MRMs) in contact with uncertain environments based on the adaptive dynamic programming (ADP) algorithm. The dynamic model of MRMs is formulated via joint torque feedback technique that is deployed for each joint module to design the model compensation controller. An uncertainty decomposition-based robust control is developed to compensate the model uncertainties, and then, the robust optimal control problem of the MRM system is transformed into a two-player zero-sum optimal control one. According to the ADP algorithm, the Hamilton-Jacobi-Isaacs equation can be solved by establishing action and critic neural networks, thus making the derivation of the approximate optimal control policy feasible. Based on the Lyapunov theory, the closed-loop robotic system is proved to be asymptotic stable under the developed decentralized control method. Finally, experiments are conducted to verify the effectiveness and advantages of the proposed method.

关键词： Modular robot manipulators adaptive dynamic programming Decentralized control Optimal control Zero-sum game

来源：评论

学校读者我要写书评

暂无评论

Event-triggered single-network ADP method for constrained optimal tracking control of continuous-time non-linear systems

引用

APPLIED MATHEMATICS AND COMPUTATION 2019年 352卷 220-234页

作者： Cui, Lili Xie, Xiangpeng Wang, Xiaowei Luo, Yanhong Liu, Jingbo Shenyang Normal Univ Shenyang Liaoning Peoples R China Nanjing Univ Posts & Telecommun Nanjing Jiangsu Peoples R China Northeastern Univ Shenyang Liaoning Peoples R China State Grid Liaoning Elect Power Co Ltd Econ Technol Res Inst Shenyang Liaoning Peoples R China

This paper investigates the optimal tracking control problem (OTCP) for continuous-time non-linear systems with input constraints. A novel event-triggered single-network adaptive dynamic programming (ADP) method is proposed to obtain the solution of constrained OTCP. By constructing an augmented system and introducing a novel discounted non-quadratic cost function, an event-triggered constrained tracking Hamilton-Jacobi-Bellman equation is formulated. Then, only a critic neural network (NN) is employed to learn the optimal value function and further obtain the optimal tracking controller, which enables the architecture of ADP implementation to be simpler. And a novel NN weights updating law is constructed, by which the restriction of initial admissible control is removed. Based on the Lyapunov theory, the convergence of critic NN weights and the stability of closed-loop system are demonstrated. The derived optimal tracking controller is updated only at the event-triggered instants decided by the designed event-triggered condition. Therefore, the communication burden can be reduced effectively. Finally, two simulation examples are given to verify the effectiveness of proposed method. (C) 2019 Elsevier Inc. All rights reserved.

关键词： adaptive dynamic programming Tracking control Input constraints Event-triggered

来源：评论

学校读者我要写书评

暂无评论

Reliable control of cyber-physical systems under sensor and actuator attacks: An identifier-critic based integral sliding-mode control approach

引用

NEUROCOMPUTING 2019年 361卷 229-242页

作者： Huang, Xin Dong, Jiuxiang Northeastern Univ State Key Lab Synthet Automat Proc Ind Shenyang 110819 Liaoning Peoples R China Northeastern Univ Coll Informat Sci & Engn Shenyang 110819 Liaoning Peoples R China Northeastern Univ Shenyang Minist Educ Key Lab Vibrat & Control Aeroprop Syst Shenyang 110819 Liaoning Peoples R China

This study focuses on reliable control problems for cyber-physical systems (CPSs), described by linear continuous time-invariant systems, under a class of time-varying state-dependent sensor and actuator attacks. According to the state-dependent property of sensor attacks, the original reliable control problem is converted into a problem to stabilise a completely unknown time-varying system with the actuator attacks. Firstly, a dynamic neural network (NN) identifier with an indicator-function term is developed to model the unknown system dynamics. Due to the indicator term, the input distribution matrix of the identifier is guaranteed to be of full column rank. It provides a technical condition for designing an effective identifier-based integral sliding-mode (I-SM) attack compensator. It is shown that, under the proposed adaptive law, the identification errors and NN weight estimation errors are uniformly ultimately bounded. Furthermore, an identifier-critic based I-SM controller, which consists of an ADP-based controller and an adaptive I-SM attack compensator, is online learned using real-time control input and compromised sensor data, such that the unknown dynamics with the actuator attacks is stable with a nearly optimal performance. Meanwhile, it is shown that, the developed control approach ensures that the states in CPSs under sensor and actuator attacks converge to a compact set around zero. Finally, the effectiveness of the proposed one is verified by two illustrative examples. (C) 2019 Elsevier B.V. All rights reserved.

关键词： Cyber-physical systems Cyber security Reliable control Sensor and actuator attacks Integral sliding-mode control adaptive dynamic programming

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：