检索结果-内蒙古大学图书馆

ieee International symposium on Approximate dynamic programming and reinforcement learning

作者： Liu, Zhenzhen Elhanany, Itamar Univ Tennessee Dept Elect & Comp Engn Knoxville TN 37996 USA

ISBN: (纸本)9781424407064

This paper presents a framework for obtaining an optimal policy in model-free Partially Observable Markov Decision Problems (POMDPs) using a recurrent neural network (RNN). A Q-function approximation approach is taken, utilizing a novel RNN architecture with computation and storage requirements that are dramatically reduced when compared to existing schemes. A scalable online training algorithm, derived from the real-time recurrent learning (RTRL) algorithm, is employed. Moreover, stochastic meta-descent (SMD), an adaptive step size scheme for stochastic gradient-descent problems, is utilized as means of incorporating curvature information to accelerate the learning process. We consider case studies of POMDPs where state information is not directly available to the agent. Particularly, we investigate scenarios in which the agent receives indentical observations for multiple states, thereby relying on temporal dependencies captured by the RNN to obtain the optimal policy. Simulation results illustrate the effectiveness of the approach along with substantial improvement in convergence rate when compared to existing schemes. Index Terms-Recurrent neural networks, real-time recurrent learning (RTRL), constraint optimization.

关键词： Constraint optimization Real-lime recurrent learning (RTRL) Recurrent neural networks

来源：评论

学校读者我要写书评

暂无评论

A Two Stage learning Technique for Dual learning in the Pursuit-Evasion Differential Game

A Two Stage Learning Technique for Dual Learning in the Purs...

引用

ieee symposium on adaptive dynamic programming and reinforcement learning (ADPRL)

作者： Al-Talabi, Ahmad A. Schwartz, Howard M. Carleton Univ Dept Syst & Comp Engn 1125 Colonel By Dr Ottawa ON K1S 5B6 Canada Univ Baghdad Al Khwarizmi Coll Engn Mechatron Engn Dept Baghdad Iraq

ISBN: (纸本)9781479945528

This paper addresses the case of dual learning in the pursuit-evasion (PE) differential game and examines how fast the players can learn their default control strategies. The players should learn their default control strategies simultaneously by interacting with each other. Each player's learning process depends on the rewards received from its environment. The learning process is implemented using a two stage learning algorithm that combines the particle swarm optimization (PSO)-based fuzzy logic control (FLC) algorithm with the Q-learning fuzzy inference system (QFIS) algorithm. The PSO algorithm is used as a global optimizer to autonomously tune the parameters of a fuzzy logic controller whereas the QFIS algorithm is used as a local optimizer. The two stage learning algorithm is compared through simulation with the default control strategy, the PSO-based FLC algorithm, and the QFIS algorithm. Simulation results show that the players are able to learn their default control strategies. Also, it shows that the two stage learning algorithm outperforms the PSO-based FLC algorithm and the QFIS algorithm with respect to the learning time.

关键词： control system analysis computing fuzzy control fuzzy reasoning game theory learning (artificial intelligence) particle swarm optimisation FLC PE PSO Q-learning fuzzy inference system algorithm QFIS default control strategies dual learning fuzzy logic controller global optimizer particle swarm optimization based fuzzy logic control algorithm pursuit-evasion differential game two stage learning technique Approximation algorithms Fuzzy logic Games Inference algorithms Sociology Statistics Tuning control system analysis computing Game theory fuzzy logic controller Inference algorithms Particle swarm optimization Fuzzy control Sociology fuzzy reasoning Approximation algorithms tuning parametric subharmonic oscillator Fuzzy logic Polyethylenes Players

来源：评论

学校读者我要写书评

暂无评论

Event-Driven H_∞-Constrained Control Using adaptive Critic learning

引用

ieee TRANSACTIONS ON CYBERNETICS 2021年第10期51卷 4860-4872页

作者： Yang, Xiong He, Haibo Tianjin Univ Sch Elect & Informat Engn Tianjin 300072 Peoples R China Univ Rhode Isl Dept Elect Comp & Biomed Engn Kingston RI 02881 USA

This article considers an event-driven H-infinity control problem of continuous-time nonlinear systems with asymmetric input constraints. Initially, the H-infinity-constrained control problem is converted into a two-person zero-sum game with the discounted nonquadratic cost function. Then, we present the event-driven Hamilton-Jacobi-Isaacs equation (HJIE) associated with the two-person zero-sum game. Meanwhile, we develop a novel event-triggering condition making Zeno behavior excluded. The present event-triggering condition differs from the existing literature in that it can make the triggering threshold non-negative without the requirement of properly selecting the prescribed level of disturbance attenuation. After that, under the framework of adaptive critic learning, we use a single critic network to solve the event-driven HJIE and tune its weight parameters by using historical and instantaneous state data simultaneously. Based on the Lyapunov approach, we demonstrate that the uniform ultimate boundedness of all the signals in the closed-loop system is guaranteed. Finally, simulations of a nonlinear plant are presented to validate the developed event-driven H-infinity control strategy.

关键词： adaptive critic learning (ACL) adaptive dynamic programming (ADP) asymmetric constraints event-driven H-infinity control reinforcement learning (RL)

来源：评论

学校读者我要写书评

暂无评论

Computing and Communication Cost-Aware Service Migration Enabled by Transfer reinforcement learning for dynamic Vehicular Edge Computing Networks

引用

ieee TRANSACTIONS ON MOBILE COMPUTING 2024年第1期23卷 257-269页

作者： Peng, Yan Tang, Xiaogang Zhou, Yiqing Li, Jintao Qi, Yanli Liu, Ling Lin, Hai Inst Comp Technol State Key Lab Processors Beijing 100190 Peoples R China Beijing Key Lab Mobile Comp & Pervas Device Beijing 100190 Peoples R China Space Engn Univ Sch Aerosp Informat Beijing 100015 Peoples R China

Due to the high mobility of vehicles, service migration is inevitable in vehicular edge computing (VEC) networks. Frequent service migrations incur prohibitive migration cost including the computing cost (e.g., increased computing delay) and communication cost (e.g., occupied backhaul bandwidth). Yet existing service migration schemes are usually designed without considering the impact of the computing cost. This paper considers the impact of computing and communication cost jointly, and proposes a computing and communication cost-aware service migration scheme for VEC networks (i.e., CA-migration). Taking the service delay as a QoS metric for VEC networks, this paper formulates a migration optimization problem aiming to maximize the services' satisfaction degree of delay (i.e., the probability that the service delay is smaller than the service delay requirement), where both the communication cost and computing cost affect the services' satisfaction degree. Since the optimization problem is a constrained non-linear integer programming problem, it is difficult to solve. Moreover, the VEC networks are highly dynamic. Thus, a fast transfer reinforcement learning (fast-TRL) method combining transfer learning and reinforcement learning is proposed to provide an adaptive service migration scheme in dynamic VEC networks. Simulation results show that compared with existing schemes, the proposed CA-migration scheme can increase the satisfaction degree by up to 30%, and needs 25% less training time to obtain the optimal service migration policy.

关键词： Vehicular edge computing service migration computing cost services' satisfaction degree fast transfer reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Hamiltonian-Driven adaptive dynamic programming With Approximation Errors

引用

ieee TRANSACTIONS ON CYBERNETICS 2022年第12期52卷 13762-13773页

作者： Yang, Yongliang Modares, Hamidreza Vamvoudakis, Kyriakos G. He, Wei Xu, Cheng-Zhong Wunsch, Donald C. Univ Sci & Technol Beijing Sch Automat & Elect Engn Beijing 100083 Peoples R China Univ Sci & Technol Beijing Inst Artificial Intelligence Beijing 100083 Peoples R China Michigan State Univ Mech Engn Dept E Lansing MI 48824 USA Georgia Tech Daniel Guggenheim Sch Aerosp Engn Atlanta GA 30332 USA Univ Macau Fac Sci & Technol State Key Lab Internet Things Smart City Macau Peoples R China Missouri Univ Sci & Technol Dept Elect & Comp Engn Rolla MO 65401 USA

In this article, we consider an iterative adaptive dynamic programming (ADP) algorithm within the Hamiltonian-driven framework to solve the Hamilton-Jacobi-Bellman (HJB) equation for the infinite-horizon optimal control problem in continuous time for nonlinear systems. First, a novel function, ``min-Hamiltonian,'' is defined to capture the fundamental properties of the classical Hamiltonian. It is shown that both the HJB equation and the policy iteration (PI) algorithm can be formulated in terms of the min-Hamiltonian within the Hamiltonian-driven framework. Moreover, we develop an iterative ADP algorithm that takes into consideration the approximation errors during the policy evaluation step. We then derive a sufficient condition on the iterative value gradient to guarantee closed-loop stability of the equilibrium point as well as convergence to the optimal value. A model-free extension based on an off-policy reinforcement learning (RL) technique is also provided. Finally, numerical results illustrate the efficacy of the proposed framework.

关键词： Costs Mathematical model Stability analysis Approximation error Approximation algorithms dynamic programming Iterative algorithms Hamilton-Jacobi-Bellman (HJB) equation Hamiltonian-driven framework inexact adaptive dynamic programming (ADP) optimal control

来源：评论

学校读者我要写书评

暂无评论

Online adaptive learning of optimal control solutions using integral reinforcement learning

Online adaptive learning of optimal control solutions using ...

引用

作者： Vamvoudakis, Kyriakos G. Vrabie, Draguna Lewis, Frank L. Automation and Robotics Research Institute University of Texas at Arlington Fort Worth TX 76118 United States

ISBN: (纸本)9781424498888

In this paper we introduce an online algorithm that uses integral reinforcement knowledge for learning the continuous-time optimal control solution for nonlinear systems with infinite horizon costs and partial knowledge of the system dynamics. This algorithm is a data based approach to the solution of the Hamilton-Jacobi-Bellman equation and it does not require explicit knowledge on the system's drift dynamics. The adaptive algorithm is based on policy iteration, and it is implemented on an actor/critic structure. Both actor and critic neural networks are adapted simultaneously a persistence of excitation condition is required to guarantee convergence of the critic to the actual optimal value function. Novel tuning algorithms are given for both critic and actor networks, with extra terms in the actor tuning law being required to guarantee closed-loop dynamical stability. The convergence to the optimal controller is proven, and stability of the system is also guaranteed. Simulation examples support the theoretical result. © 2011 ieee.

关键词： adaptive algorithms

来源：评论

学校读者我要写书评

暂无评论

reinforcement learning Control of Robotic Knee With Human-in-the-Loop by Flexible Policy Iteration

引用

ieee TRANSACTIONS ON NEURAL NETWORKS AND learning SYSTEMS 2022年第10期33卷 5873-5887页

作者： Gao, Xiang Si, Jennie Wen, Yue Li, Minhan Huang, He Arizona State Univ Dept Elect Comp & Energy Engn Tempe AZ 85287 USA North Carolina State Univ Dept Biomed Engn Raleigh NC 27695 USA Univ N Carolina Chapel Hill NC 27599 USA

We are motivated by the real challenges presented in a human-robot system to develop new designs that are efficient at data level and with performance guarantees, such as stability and optimality at system level. Existing approximate/adaptive dynamic programming (ADP) results that consider system performance theoretically are not readily providing practically useful learning control algorithms for this problem, and reinforcement learning (RL) algorithms that address the issue of data efficiency usually do not have performance guarantees for the controlled system. This study fills these important voids by introducing innovative features to the policy iteration algorithm. We introduce flexible policy iteration (FPI), which can flexibly and organically integrate experience replay and supplemental values from prior experience into the RL controller. We show system-level performances, including convergence of the approximate value function, (sub)optimality of the solution, and stability of the system. We demonstrate the effectiveness of the FPI via realistic simulations of the human-robot system. It is noted that the problem we face in this study may be difficult to address by design methods based on classical control theory as it is nearly impossible to obtain a customized mathematical model of a human-robot system either online or offline. The results we have obtained also indicate the great potential of RL control to solving realistic and challenging problems with high-dimensional control inputs.

关键词： Robots Impedance Tuning Prosthetics Knee Erbium Legged locomotion adaptive optimal control data- and time-efficient learning flexible policy iteration (FPI) human-in-the-loop reinforcement learning (RL) robotic knee

来源：评论

学校读者我要写书评

暂无评论

adaptive Configuration with Deep reinforcement learning in Software-Defined Time-Sensitive Networking

Adaptive Configuration with Deep Reinforcement Learning in S...

引用

ieee/IFIP Network Operations and Management symposium (NOMS)

作者： Guo, Mengjie Shou, Guochu Liu, Yaqiong Hu, Yihong Beijing Univ Posts & Telecommun Sch Informat & Commun Engn Beijing Peoples R China

ISBN: (纸本)9798350327939;9798350327946

Time-sensitive networking (TSN) is very appealing to industrial networks due to its support for deterministic transmission based on Ethernet. The implementation of determinism typically demands for precise configuration on each output port of a TSN switch, which is complex and time-consuming. Moreover, many emerging industrial applications bring dynamic scenarios (e.g., in real-time Internet of Things), thus the configurations should change adaptively as application requirements change to provide continued determinism. In this paper, we propose a deep reinforcement learning (DRL) based adaptive configuration scheme in Software-defined time-sensitive networking (SD-TSN). The SD-TSN is a network architecture that integrates the determinism guarantees of TSN and flexible network management of software-defined networking (SDN). Based on the capability of SD-TSN, the proposed configuration scheme exploits DRL to learn from interacting with the environment for adaptive configuration. Experimental results demonstrate the effectiveness of our scheme in dynamic scenarios.

关键词： Software-defined time-sensitive networking (SD-TSN) deep reinforcement learning (DRL) adaptive configuration

来源：评论

学校读者我要写书评

暂无评论

Mutual reinforcement learning with Heterogenous Agents 7

Mutual Reinforcement Learning with Heterogenous Agents

引用

7th ieee International Conference on Smart Computing (SMARTCOMP)

作者： Reid, Cameron Mukhopadhyay, Snehasis Indiana Univ Purdue Univ Sch Comp & Informat Sci Indianapolis IN 46202 USA

ISBN: (纸本)9781665412520

Mutual learning is an emerging technique for allowing intelligent systems to learn from each other, giving rise to improved performance. In this paper, we explore mutual reinforcement learning between systems which use very different learning algorithms. In particular, we present an algorithm which allows two agents, one using Q-learning and another using adaptive dynamic programming, to share learned knowledge. We discuss how these agents negotiate the relative importance of knowledge they receive from other agents, and we present results that show how this affects the learning process.

关键词： Training Heuristic algorithms Conferences reinforcement learning dynamic programming Intelligent systems

来源：评论

学校读者我要写书评

暂无评论

Approximate Nash Solutions for Multiplayer Mixed-Zero-Sum Game With reinforcement learning

引用

ieee TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS 2019年第12期49卷 2739-2750页

作者： Lv, Yongfeng Ren, Xuemei Beijing Inst Technol Sch Automat Beijing 100081 Peoples R China

Inspired by Nash game theory, a multiplayer mixed-zero-sum (MZS) nonlinear game considering both two situations [zero-sum and nonzero-sum (NZS) Nash games] is proposed in this paper. A synchronous reinforcement learning (RL) scheme based on the identifier-critic structure is developed to learn the Nash equilibrium solution of the proposed MZS game. First, the MZS game formulation is presented, where the performance indexes for players 1 to N - 1 and N NZS Nash game are presented, and another performance index for players N and N + 1 zero-sum game is presented, such that player N cooperates with players 1 to N - 1, while competes with player N + 1, which leads to a Nash equilibrium of all players. A single-layer neural network (NN) is then used to approximate the unknown dynamics of the nonlinear game system. Finally, an RL scheme based on NNs is developed to learn the optimal performance indexes, which can be used to produce the optimal control policy of every player such that Nash equilibrium can be obtained. Thus, the widely used actor NN in RL literature is not needed. To this end, a recently proposed adaptive law is used to estimate the unknown identifier coefficient vectors, and an improved adaptive law with the error performance index is further developed to update the critic coefficient vectors. Both linear and nonlinear simulations are presented to demonstrate the existence of Nash equilibrium for MZS game and performance of the proposed algorithm.

关键词： Approximate dynamic programming (ADP) Nash games neural networks (NNs) reinforcement learning (RL) system identification

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：