检索结果-内蒙古大学图书馆

Policy Iteration adaptive dynamic programming Algorithm for Discrete-Time Nonlinear Systems

ieee TRANSACTIONS ON NEURAL NETWORKS AND learning SYSTEMS 2014年第3期25卷 621-634页

作者： Liu, Derong Wei, Qinglai Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China

This paper is concerned with a new discrete-time policy iteration adaptive dynamic programming (ADP) method for solving the infinite horizon optimal control problem of nonlinear systems. The idea is to use an iterative ADP technique to obtain the iterative control law, which optimizes the iterative performance index function. The main contribution of this paper is to analyze the convergence and stability properties of policy iteration method for discrete-time nonlinear systems for the first time. It shows that the iterative performance index function is nonincreasingly convergent to the optimal solution of the Hamilton-Jacobi-Bellman equation. It is also proven that any of the iterative control laws can stabilize the nonlinear systems. Neural networks are used to approximate the performance index function and compute the optimal control law, respectively, for facilitating the implementation of the iterative ADP algorithm, where the convergence of the weight matrices is analyzed. Finally, the numerical results and analysis are presented to illustrate the performance of the developed method.

关键词： adaptive critic designs adaptive dynamic programming (ADP) approximate dynamic programming discrete-time policy iteration neural networks neurodynamic programming nonlinear systems optimal control reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

adaptive dynamic programming for Discrete-time LQR Optimal Tracking Control Problems with Unknown dynamics

Adaptive Dynamic Programming for Discrete-time LQR Optimal T...

引用

ieee symposium on adaptive dynamic programming and reinforcement learning (ADPRL)

作者： Liu, Yang Luo, Yanhong Zhang, Huaguang Northeastern Univ Sch Informat Sci & Engn Shenyang 110819 Liaoning Peoples R China

ISBN: (纸本)9781479945528

In this paper, an optimal tracking control approach based on adaptive dynamic programming (ADP) algorithm is proposed to solve the linear quadratic regulation (LQR) problems for unknown discrete-time systems in an online fashion. First, we convert the optimal tracking problem into designing infinite-horizon optimal regulator for the tracking error dynamics based on the system transformation. Then we expand the error state equation by the history data of control and state. The iterative ADP algorithm of policy iteration (PI) and value iteration (VI) are introduced to solve the value function of the controlled system. It is shown that the proposed ADP algorithm solves the LQR without requiring any knowledge of the system dynamics. The simulation results show the convergence and effectiveness of the proposed control scheme.

关键词： Digital control systems

来源：评论

学校读者我要写书评

暂无评论

adaptive Multi-Step Evaluation Design With Stability Guarantee for Discrete-Time Optimal learning Control

引用

ieee/CAA Journal of Automatica Sinica 2023年第9期10卷 1797-1809页

作者： Ding Wang Jiangyu Wang Mingming Zhao Peng Xin Junfei Qiao IEEE Faculty of Information Technology the Beijing Key Laboratory of Computational Intelligence and Intelligent Systemthe Beijing Laboratory of Smart Environmental Protectionand the Beijing Institute of Artificial IntelligenceBeijing University of TechnologyBeijing 100124China

This paper is concerned with a novel integrated multi-step heuristic dynamic programming(MsHDP)algorithm for solving optimal control *** is shown that,initialized by the zero cost function,MsHDP can converge to the optimal solution of the Hamilton-Jacobi-Bellman(HJB)***,the stability of the system is analyzed using control policies generated by ***,a general stability criterion is designed to determine the admissibility of the current control *** is,the criterion is applicable not only to traditional value iteration and policy iteration but also to ***,based on the convergence and the stability criterion,the integrated MsHDP algorithm using immature control policies is developed to accelerate learning efficiency ***,actor-critic is utilized to implement the integrated MsHDP scheme,where neural networks are used to evaluate and improve the iterative policy as the parameter ***,two simulation examples are given to demonstrate that the learning effectiveness of the integrated MsHDP scheme surpasses those of other fixed or integrated methods.

关键词： adaptive critic artificial neural networks Hamilton-Jacobi-Bellman(HJB)equation multi-step heuristic dynamic programming multi-step reinforcement learning optimal control

来源：评论

学校读者我要写书评

暂无评论

adaptive dynamic programming and adaptive Optimal Output Regulation of Linear Systems

引用

ieee TRANSACTIONS ON AUTOMATIC CONTROL 2016年第12期61卷 4164-4169页

作者： Gao, Weinan Jiang, Zhong-Ping NYU Tandon Sch Engn Dept Elect & Comp Engn Brooklyn NY 11201 USA

This note studies the adaptive optimal output regulation problem for continuous-time linear systems, which aims to achieve asymptotic tracking and disturbance rejection by minimizing some predefined costs. reinforcement learning and adaptive dynamic programming techniques are employed to compute an approximated optimal controller using input/partial-state data despite unknown system dynamics and unmeasurable disturbance. Rigorous stability analysis shows that the proposed controller exponentially stabilizes the closed-loop system and the output of the plant asymptotically tracks the given reference signal. Simulation results on a LCL coupled inverter-based distributed generation system demonstrate the effectiveness of the proposed approach.

关键词： adaptive control approximate/adaptive dynamic programming (ADP) optimal control output regulation

来源：评论

学校读者我要写书评

暂无评论

Iterative Local dynamic programming

Iterative Local Dynamic Programming

引用

ieee symposium on adaptive dynamic programming and reinforcement learning

作者： Todorov, Emanuel Tassa, Yuval Univ Calif San Diego Dept Cognit Sci La Jolla CA 92093 USA Hebrew Univ Jerusalem Ctr Neural Computat IL-91905 Jerusalem Israel

ISBN: (纸本)9781424427611

We develop an iterative local dynamic programming method (iLDP) applicable to stochastic optimal control problems in continuous high-dimensional state and action spaces. Such problems are common in the control of biological movement, but cannot be handled by existing methods. iLDP can be considered a generalization of Differential dynamic programming, inasmuch as: (a) we use general basis functions rather than quadratics to approximate the optimal value function;(b) we introduce a collocation method that dispenses with explicit differentiation of the cost and dynamics and ties iLDP to the Unscented Kalman filter;(c) we adapt the local function approximator to the propagated state covariance, thus increasing accuracy at more likely states. Convergence is similar to quasi-Netwon methods. We illustrate iLDP on several problems including the "swimmer" dynamical system which has 14 state and 4 control variables.

关键词： dynamical systems

来源：评论

学校读者我要写书评

暂无评论

The QV Family Compared to Other reinforcement learning Algorithms

The QV Family Compared to Other Reinforcement Learning Algor...

引用

ieee symposium on adaptive dynamic programming and reinforcement learning

作者： Wiering, Marco A. van Hasselt, Hado Univ Groningen Dept Artificial Intelligence NL-9700 AB Groningen Netherlands Univ Utrecht Intelligent Syst Grp NL-3508 TC Utrecht Netherlands

ISBN: (纸本)9781424427611

This paper describes several new online model-free reinforcement learning (RL) algorithms. We designed three new reinforcement algorithms, namely: QV2, QVMAX, and QV-MAX2, that are all based on the QV-learning algorithm, but in contrary to QV-learning, QVMAX and QVMAX2 are off-policy RL algorithms and QV2 is a new on-policy RL algorithm. We experimentally compare these algorithms to a large number of different RL algorithms, namely: Q-learning, Sarsa, R-learning, Actor-Critic, QV-learning, and ACLA. We show experiments on five maze problems of varying complexity. Furthermore, we show experimental results on the cart pole balancing problem. The results show that for different problems, there can be large performance differences between the different algorithms, and that there is not a single RL algorithm that always performs best, although on average QV-learning scores highest.

关键词： learning algorithms

来源：评论

学校读者我要写书评

暂无评论

reinforcement learning Based adaptive Blocklength and MCS for Optimizing Age Violation Probability

引用

ieee ACCESS 2023年 11卷 122411-122425页

作者： Ozkaya, Aysenur Topbas, Ahsen Ceran, Elif Tugce Aselsan Inc TR-06200 Ankara Turkiye Middle East Tech Univ Dept Elect & Elect Engn TR-06800 Ankara Turkiye

As a measure of the freshness of data, Age of Information (AoI) has become an essential performance metric in status update applications with stringent timeliness constraints. This study employs adaptive strategies to minimize the novel, information freshness-based performance metric age violation probability (AVP), the probability of the instantaneous age exceeding a predefined constraint, in short packet communications (SPC). AVP can be considered one of the key performance indicators (KPIs) in 5G Ultra-Reliable Low Latency Communications (URLLC), and it is expected to gain more importance in 6G technologies, especially in extreme URLLC (xURLLC). Two distinct approaches are considered: the first focuses on adaptively selecting the blocklengths with either imperfect or missing channel state information exploiting finite blocklength theory approximations. The second involves dynamically choosing the modulation and coding scheme (MCS) to minimize the AVP under stringent timeliness constraints and non-asymptotic information theory bounds. In the context of adaptive blocklength selection, state-aggregated value iteration, Q-learning algorithms, and finite blocklength theory approximations are leveraged to adjust blocklengths to achieve low age violation probabilities adaptively. The simulation results highlight the effectiveness of these algorithms in minimizing age violation probabilities compared to the fixed blocklengths under varying channel conditions. Additionally, constructing a deep reinforcement learning (DRL) framework, we propose a deep Q-network policy for the dynamic selection of the modulation and coding scheme among the available MCSs defined for URLLC systems. Through comprehensive simulations, we demonstrate the superiority of the proposed adaptive methods over traditional benchmark methods.

关键词： Measurement Modulation Encoding Ultra reliable low latency communication Fading channels Throughput Minimization Information age reinforcement learning dynamic programming Age of information reinforcement learning dynamic programming finite blocklength adaptive modulation and coding

来源：评论

学校读者我要写书评

暂无评论

adaptive dynamic programming for Finite-Horizon Optimal Control of Discrete-Time Nonlinear Systems with ε-Error Bound

引用

ieee TRANSACTIONS ON NEURAL NETWORKS 2011年第1期22卷 24-36页

作者： Wang, Fei-Yue Jin, Ning Liu, Derong Wei, Qinglai Chinese Acad Sci Inst Automat Key Lab Complex Syst & Intelligence Sci Beijing 100190 Peoples R China Univ Illinois Dept Elect & Comp Engn Chicago IL 60607 USA

In this paper, we study the finite-horizon optimal control problem for discrete-time nonlinear systems using the adaptive dynamic programming (ADP) approach. The idea is to use an iterative ADP algorithm to obtain the optimal control law which makes the performance index function close to the greatest lower bound of all performance indices within an epsilon-error bound. The optimal number of control steps can also be obtained by the proposed ADP algorithms. A convergence analysis of the proposed ADP algorithms in terms of performance index function and control policy is made. In order to facilitate the implementation of the iterative ADP algorithms, neural networks are used for approximating the performance index function, computing the optimal control policy, and modeling the nonlinear system. Finally, two simulation examples are employed to illustrate the applicability of the proposed method.

关键词： adaptive critic designs adaptive dynamic programming approximate dynamic programming learning control neural control neural dynamic programming optimal control reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Supervised adaptive dynamic programming based adaptive cruise control

Supervised adaptive dynamic programming based adaptive cruis...

引用

symposium Series on Computational Intelligence, ieee SSCI2011 - 2011 ieee symposium on adaptive dynamic programming and reinforcement learning, ADPRL 2011

作者： Zhao, Dongbin Hu, Zhaohui Key Laboratory of Complex Systems and Intelligence Science Institute of Automation Chinese Academy of Sciences Beijing 100190 China

ISBN: (纸本)9781424498888

This paper proposes a supervised adaptive dynamic programming (SADP) algorithm for the full range adaptive cruise control (ACC) system. The full range ACC system considers both the ACC situation in highway system and the stop and go (SG) situation in urban street way system. It can autonomously drive the host vehicle with desired speed and distance to the preceding vehicle in both situations. A traditional adaptive dynamic programming (ADP) algorithm is suited for this problem, but it suffers from the low learning efficiency. We propose the concept of inducing range to construct the supervisor and finally formulate the SADP algorithm, which greatly speeds up the learning efficiency. Several driving scenarios are designed and tested with the trained controller compared to traditional ones by simulation results, showing that trained SADP performs very well in all the scenarios, so that it provides an effective approach for the full range ACC problem. © 2011 ieee.

关键词： adaptive cruise control

来源：评论

学校读者我要写书评

暂无评论

adaptive learning in Tracking Control Based on the Dual Critic Network Design

引用

ieee TRANSACTIONS ON NEURAL NETWORKS AND learning SYSTEMS 2013年第6期24卷 913-928页

作者： Ni, Zhen He, Haibo Wen, Jinyu Univ Rhode Isl Dept Elect Comp & Biomed Engn Kingston RI 02881 USA Huazhong Univ Sci & Technol Coll Elect Elect & Engn Wuhan 430074 Peoples R China

In this paper, we present a new adaptive dynamic programming approach by integrating a reference network that provides an internal goal representation to help the systems learning and optimization. Specifically, we build the reference network on top of the critic network to form a dual critic network design that contains the detailed internal goal representation to help approximate the value function. This internal goal signal, working as the reinforcement signal for the critic network in our design, is adaptively generated by the reference network and can also be adjusted automatically. In this way, we provide an alternative choice rather than crafting the reinforcement signal manually from prior knowledge. In this paper, we adopt the online action-dependent heuristic dynamic programming (ADHDP) design and provide the detailed design of the dual critic network structure. Detailed Lyapunov stability analysis for our proposed approach is presented to support the proposed structure from a theoretical point of view. Furthermore, we also develop a virtual reality platform to demonstrate the real-time simulation of our approach under different disturbance situations. The overall adaptive learning performance has been tested on two tracking control benchmarks with a tracking filter. For comparative studies, we also present the tracking performance with the typical ADHDP, and the simulation results justify the improved performance with our approach.

关键词： adaptive critic design (ACD) adaptive dynamic programming (ADP) internal goal lyapunov stability analysis online learning reinforcement learning tracking control virtual reality

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：