检索结果-内蒙古大学图书馆

International Joint Conference on Neural Networks (IJCNN)

作者： Zhao, Qiming Xu, Hao Jagannathan, S. Missouri Univ S&T Dept Elect & Comp Engn Rolla MO 65409 USA

ISBN: (纸本)9781467314909

In this paper, the Bellman equation is used to solve the stochastic optimal control of unknown linear discrete-time system with communication imperfections including random delays, packet losses and quantization. A dynamic quantizer for the sensor measurements is proposed which essentially provides system states to the controller. To eliminate the effect of the quantization error, the dynamics of the quantization error bound and an update law for tuning its range are derived. Subsequently, by using adaptive dynamic programming technique, the infinite horizon optimal regulation of the uncertain NCS is solved in a forward-in-time manner without using value and/or policy iterations by using Q-function and reinforcement learning. The asymptotic stability of the closed-loop system is verified by standard Lyapunov stability theory. Finally, the effectiveness of the proposed method is verified by simulation results.

关键词： Networked Control System Quantization adaptive dynamic programming Optimal Control

来源：评论

学校读者我要写书评

暂无评论

Observer-Based adaptive Synchronization Control of Unknown Discrete-Time Nonlinear Heterogeneous Systems

引用

ieee TRANSACTIONS ON NEURAL NETWORKS AND learning SYSTEMS 2022年第2期33卷 681-693页

作者： Fu, Hao Chen, Xin Wang, Wei Wu, Min China Univ Geosci Sch Automat Wuhan 430074 Peoples R China Hubei Key Lab Adv Control & Intelligent Automat C Wuhan 430074 Peoples R China WISDRI Engn & Res Inc Ltd Wuhan 430223 Peoples R China

This article is concerned with the optimal synchronization problem for discrete-time nonlinear heterogeneous multiagent systems (MASs) with an active leader. To overcome the difficulty in the derivation of the optimal control protocols for these systems, we develop an observer-based adaptive synchronization control approach, including the designs of a distributed observer and a distributed model reference adaptive controller with no prior knowledge of all agents' dynamics. To begin with, for the purpose of estimating the state of a nonlinear active leader for each follower, an adaptive neural network distributed observer is designed. Such an observer serves as a reference model in the distributed model reference adaptive control (MRAC). Then, a reinforcement learning-based distributed MRAC algorithm is presented to make every follower track its corresponding reference model on behavior in real time. In this algorithm, a distributed actor-critic network is employed to approximate the optimal distributed control protocols and the cost function. Through convergence analysis, the overall observer estimation error, the model reference tracking error, and the weight estimation errors are proved to be uniformly ultimately bounded. The developed approach further achieves the synchronization by means of synthesizing these results. The effectiveness of the developed approach is verified through a numerical example.

关键词： Synchronization Observers Artificial neural networks Adaptation models Mathematical model Protocols adaptive systems adaptive dynamic programming (ADP) discrete time multiagent systems (MASs) observer optimal synchronization

来源：评论

学校读者我要写书评

暂无评论

Event-Triggered Optimal Neuro-Controller Design With reinforcement learning for Unknown Nonlinear Systems

引用

ieee TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS 2019年第9期49卷 1866-1878页

作者： Yang, Xiong He, Haibo Liu, Derong Univ Rhode Isl Dept Elect Comp & Biomed Engn Kingston RI 02881 USA Tianjin Univ Sch Elect & Informat Engn Tianjin 300072 Peoples R China Guangdong Univ Technol Sch Automat Guangzhou 510006 Guangdong Peoples R China

This paper develops an optimal control scheme for continuous-time unknown nonlinear systems using the event-triggering mechanism. Different from designing controllers using the time-triggering mechanism, the event-triggered controller is updated only when the system state deviates more than a certain threshold from a prescribed value. To obtain the event-triggered optimal controller, we develop an identifier-critic architecture under the framework of reinforcement learning. The identifier network, composed of a feedforward neural network (FNN), aims to derive the knowledge of unknown system dynamics, and the critic network, constituted of an FNN, intends to derive the event-triggered optimal controller. The identifier network is tuned via the combination of a standard back-propagation algorithm and an e-modification method, and the critic network is updated using a modification of the gradient descent method. By introducing an additional stability term to update the critic network, the initial admissible control is no longer required. Meanwhile, by using historical and instantaneous state data together, the persistence of excitation condition is relaxed. A stability analysis of the closed-loop system is provided based on the Lyapunov method. The effectiveness of the proposed designs is illustrated through simulations of a nonlinear example and a single link robot arm system.

关键词： adaptive dynamic programming (ADP) event-triggered control neural networks (NNs) nonlinear systems optimal control reinforcement learning (RL)

来源：评论

学校读者我要写书评

暂无评论

Multi-Kernel Online reinforcement learning for Path Tracking Control of Intelligent Vehicles

引用

ieee TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS 2021年第11期51卷 6962-6975页

作者： Liu, Jiahang Huang, Zhenhua Xu, Xin Zhang, Xinglong Sun, Shiliang Li, Dazi Natl Univ Def Technol Coll Intelligence Sci & Technol Changsha 410073 Peoples R China Rocket Engn Design & Res Inst Beijing 100011 Peoples R China East China Normal Univ Dept Comp Sci & Technol Shanghai 200241 Peoples R China Beijing Univ Chem Technol Coll Informat Sci & Technol Beijing 100029 Peoples R China

Path tracking control of intelligent vehicles has to deal with the difficulties of model uncertainties and nonlinearities. As a class of adaptive optimal control methods, reinforcement learning (RL) has received increasing attention in solving difficult control problems. However, feature representation and online learning ability are two major problems to be solved for learning control of uncertain dynamic systems. In this article, we propose a multi-kernel online RL approach for path tracking control of intelligent vehicles. In the proposed approach, a multiple kernel feature learning framework is designed for online learning control based on dual heuristic programming (DHP) and the new online learning control algorithm is called multi-kernel DHP (MKDHP). In MKDHP, instead of the expert knowledge for selecting and fine-tuning of a suitable kernel function, only a set of basic kernel functions is required to be predefined and the multi-kernel features can be learned for value function approximation in the critic. The simulation studies on path tracking control for intelligent vehicles have been conducted under S-curve and urban road conditions. The results demonstrated that compared with other typical path tracking controllers for intelligent vehicles, such as the linear quadratic regulator (LQR), the pure pursuit controller and the ribbon-based controller, the proposed multi-kernel learning controller can achieve better performance in terms of tracking precision and smoothness.

关键词： Kernel Intelligent vehicles Vehicle dynamics Adaptation models Optimal control Heuristic algorithms Approximation algorithms Intelligent vehicle multiple kernel learning (MKL) path tracking reinforcement learning (RL) steering control

来源：评论

学校读者我要写书评

暂无评论

adaptive dynamic programming for terminally constrained finite-horizon optimal control problems 53

Adaptive dynamic programming for terminally constrained fini...

引用

53rd ieee Annual Conference on Decision and Control (CDC)

作者： Andrews, L. Klotz, J. R. Kamalapurkar, R. Dixon, W. E. Univ Florida Dept Mech & Aerosp Engn Gainesville FL USA

ISBN: (纸本)9781467360906

adaptive dynamic programming is applied to control-affine nonlinear systems with uncertain drift dynamics to obtain a near-optimal solution to a finite-horizon optimal control problem with hard terminal constraints. A reinforcement learning-based actor-critic framework is used to approximately solve the Hamilton-Jacobi-Bellman equation, wherein critic and actor neural networks (NN) are used for approximate learning of the optimal value function and control policy, while enforcing the optimality condition resulting from the hard terminal constraint. Concurrent learning-based update laws relax the restrictive persistence of excitation requirement. A Lyapunov-based stability analysis guarantees uniformly ultimately bounded convergence of the enacted control policy to the optimal control policy.

关键词： dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Smooth and Stopping Interval Aware Driving Behavior Prediction at Un-signalized Intersection with Inverse reinforcement learning on Sequential MDPs 32

Smooth and Stopping Interval Aware Driving Behavior Predicti...

引用

32nd ieee Intelligent Vehicles symposium (IV)

作者： Yang, Shaoyu Yoshitake, Hiroshi Shino, Motoki Shimosaka, Masamichi Tokyo Inst Technol Dept Comp Sci Tokyo Japan Univ Tokyo Dept Human & Engn Environm Studies Chiba Japan

ISBN: (纸本)9781728153940

Driving behavior modeling (DBM) is widely used in the intelligent vehicle field to prevent accidents, which predicts actions that vehicles should take to optimize safe driving behaviors. According to some statistics, accidents easily happen at un-signalized intersections. Modeling driving behavior at such places is of great importance. However, current inverse reinforcement learning-based DBM methods fail to predict proper behaviors at the un-signalized intersections in the aspects of smoothness and stopping behavior by just using a single Markov decision process (MDP). We propose a novel sequential MDPs approach to model the driving behavior at the un-signalized intersections to solve the problems. Our approach decomposes the target behavior through the un-signalized intersections into three parts and models each decomposition's driving behaviors with appropriate time durations by a stopping-time-interval distribution through dynamic programming. Experiments on real driving data show that the proposed method achieved a better result and successfully improved the smoothness and stopping awareness of the planned driving path compared to the baselines.

关键词： Intelligent vehicles Computational modeling reinforcement learning Predictive models Markov processes dynamic programming Vehicle dynamics

来源：评论

学校读者我要写书评

暂无评论

Energy- and Cost-Efficient Transmission Strategy for UAV Trajectory Tracking Control: A Deep reinforcement learning Approach

引用

ieee INTERNET OF THINGS JOURNAL 2023年第10期10卷 8958-8970页

作者： Zhang, Minkai Wu, Shaohua Jiao, Jian Zhang, Ning Zhang, Qinyu Harbin Inst Technol Shenzhen Dept Elect & Informat Engn Shenzhen 518055 Peoples R China Harbin Inst Technol Shenzhen Guangdong Prov Key Lab Aerosp Commun & Networking Shenzhen 518055 Peoples R China Peng Cheng Lab Dept Broadband Commun Shenzhen 518055 Peoples R China Univ Windsor Dept Elect & Comp Engn Windsor ON N9B 3P4 Canada

this article, we consider a networked control system (NCS) with network-induced delay, in which the control center needs to control the remote unmanned aerial vehicle (UAV) to complete the trajectory tracking task. The sensor of the controlled UAV adopts the event-triggered mechanism, while the control center uses the adaptive dynamic programming (ADP) based tracking control method to generate control actions. The application of the ADP method brings new transmission options, i.e., the control center can choose to transmit control action or neural network model. Considering the fundamental tradeoff between these two transmission options with different transmission energy consumption and tracking cost, we formulate the joint optimization problem as a Markov decision process (MDP). Due to the continuous value of state in MDP, we propose the deep Q-network (DQN)-based strategy, which uses the reinforcement learning (RL) algorithm, specifically DQN. Besides, we further propose a greedy strategy by calculating the instantaneous expected cost. Simulation results show that DQN-based strategy has better performance but depends on the training process, while greedy strategy is suboptimal but easy to compute. Besides, compared with the benchmark strategies, the proposed strategies can achieve a better compromise in the long-term average energy consumption and tracking cost by adjusting the value of the weighted factor. Furthermore, by comparing the difference of transmission decisions in the proposed strategies, we show that the proper transmission sequence in DQN-based strategy can reduce the tracking cost and transmission energy at the same time.

关键词： Costs Control systems Artificial neural networks Energy consumption Autonomous aerial vehicles Internet of Things Trajectory tracking adaptive dynamic programming (ADP) networked control system (NCS) transmission strategy unmanned aerial vehicle (UAV)

来源：评论

学校读者我要写书评

暂无评论

adaptive Constrained Optimal Control Design for Data-Based Nonlinear Discrete-Time Systems With Critic-Only Structure

引用

ieee TRANSACTIONS ON NEURAL NETWORKS AND learning SYSTEMS 2018年第6期29卷 2099-2111页

作者： Luo, Biao Liu, Derong Wu, Huai-Ning Chinese Acad Sci State Key Lab Management & Control Complex Syst Inst Automat Beijing 100190 Peoples R China Guangdong Univ Technol Sch Automat Guangzhou 510006 Guangdong Peoples R China Beihang Univ Sci & Technol Aircraft Control Lab Beijing 100191 Peoples R China

reinforcement learning has proved to be a powerful tool to solve optimal control problems over the past few years. However, the data-based constrained optimal control problem of nonaffine nonlinear discrete-time systems has rarely been studied yet. To solve this problem, an adaptive optimal control approach is developed by using the value iteration-based Q-learning (VIQL) with the critic-only structure. Most of the existing constrained control methods require the use of a certain performance index and only suit for linear or affine nonlinear systems, which is unreasonable in practice. To overcome this problem, the system transformation is first introduced with the general performance index. Then, the constrained optimal control problem is converted to an unconstrained optimal control problem. By introducing the action-state value function, i.e., Q-function, the VIQL algorithm is proposed to learn the optimal Q-function of the data-based unconstrained optimal control problem. The convergence results of the VIQL algorithm are established with an easy-to-realize initial condition Q((0))(x, a) >= 0. To implement the VIQL algorithm, the critic-only structure is developed, where only one neural network is required to approximate the Q-function. The converged Q-function obtained from the critic-only VIQL method is employed to design the adaptive constrained optimal controller based on the gradient descent scheme. Finally, the effectiveness of the developed adaptive control method is tested on three examples with computer simulation.

关键词： adaptive control adaptive dynamic programming constraints critic-only data-based optimal control Q-learning

来源：评论

学校读者我要写书评

暂无评论

Online Off-Policy reinforcement learning for Optimal Control of Unknown Nonlinear Systems Using Neural Networks

引用

ieee TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS 2024年第8期54卷 5112-5122页

作者： Zhu, Liao Wei, Qinglai Guo, Ping Beijing Normal Univ Int Acad Ctr Complex Syst Zhuhai 519087 Peoples R China Beijing Normal Univ Sch Syst Sci Beijing 100875 Peoples R China Chinese Acad Sci Inst Automat State Key Lab Multimodal Artificial Intelligence S Beijing 100190 Peoples R China Univ Chinese Acad Sci Sch Artificial Intelligence Beijing 100049 Peoples R China Macau Univ Sci & Technol Inst Syst Engn Macau Peoples R China

In this article, a real-time online off-policy reinforcement learning (RL) method is developed for the optimal control problem of unknown continuous-time nonlinear systems. First, by applying the temporal difference technique to the iterative procedure of off-policy RL, the iterative value function and the iterative policy input can be learned in real-time online. It is proven that the fitting error of neural network (NN) weights is exponentially convergent in each iteration. Second, a model-free Hamilton-Jacobi-Bellman equation (MF-HJBE) is deduced by taking the limit of the iterative procedure of off-policy RL. In this manner, it not only eliminates system dynamics in the classical HJBE, but also vanishes the iteration index. By applying temporal difference to the MF-HJBE, a real-time online tuning rule is designed to learn the optimal value function and the optimal policy input. It is proven that the fitting error of NN weights caused by the real-time online tuning rule is exponentially convergent. Note that the two online tuning rules, the iterative one and the real-time one, use only current and previous state data extracted from system trajectories. Meanwhile, it is proven using the Lyapunov's direct method that the system solution is uniformly ultimately bounded. Finally, simulation results demonstrate the validity of the proffered method.

关键词： adaptive dynamic programming nonlinear systems online learning optimal control reinforcement learning (RL)

来源：评论

学校读者我要写书评

暂无评论

Online Finite-Horizon ADP Algorithm for Solving Non-Cooperative Differential Games 36

Online Finite-Horizon ADP Algorithm for Solving Non-Cooperat...

引用

36th Chinese Control and Decision Conference (CCDC)

作者： Gao, Yingning Ma, Kemao Harbin Inst Technol Control & Simulat Ctr Harbin Peoples R China

ISBN: (纸本)9798350387780;9798350387797

In this paper, online reinforcement learning solution to finite-horizon non-cooperative differential games is investigated. The main challenges are that, the Hamilton-Jacobi-Isaacs equation is time-varying, and in terms of parameter convergence, conventional learning approaches, with persistence of excitation condition satisfied, may cause undesired oscillations in the system states and deteriorate system performance, especially in finitehorizon situations. Regarding these, we develop a finite-horizon adaptive dynamic programming algorithm for continuous-time systems, where the idea of simulation of experience in concurrent learning technique is adopted to guarantee fast online parameter convergence. Boundedness of the system states and weight estimation errors is proved. Simulation results demonstrate the effectiveness of the presented method.

关键词： finite-horizon differential games reinforcement learning adaptive dynamic programming concurrent learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：