检索结果-内蒙古大学图书馆

Fractional-Order Systems Optimal Control via Actor-Critic reinforcement learning and Its Validation for Chaotic MFET

引用

ieee TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING 2024年 22卷 1173-1182页

作者： Li, Dongdong Dong, Jiuxiang Northeastern Univ Coll Informat Sci & Engn State Key Lab Synthet Automat Proc Ind Shenyang 110819 Peoples R China Northeastern Univ Key Lab Vibrat & Control Aeroprop Syst Minist Educ Shenyang 110819 Peoples R China

Since the existence of fractional order dynamics, it is difficult to obtain an optimality equation to solve for fractional-order optimal control. In this paper, a fractional Hamilton-Jacobi-Bellman (HJB) equation based on error derivative is proposed, and a corresponding online learning algorithm is designed. The scheme can handle the optimal tracking problem for the 0 < alpha <= 1 order nonlinear systems. Since the traditional quadratic cost function is unbounded at infinite time and the optimal control derived from the discounted cost function fails to stabilize the system asymptotically, a cost function based on the error derivative is proposed, which can avoid these problems, and the system is not restricted to be zero equilibrium. Then, the fractional HJB equation is derived by constructing an auxiliary signal without directly using the chain rule of differentiation. The optimality, stability and convergence of its solution are proved, and actor-critic neural networks (NNs) are established to perform the RL algorithm. Finally, the algorithm is applied to a chaotic magnetic-field electromechanical transducer (MFET) system to verify the effectiveness and advantages.

关键词： Fractional-order systems optimal control reinforcement learning (RL) neural networks adaptive dynamic programming

来源：评论

学校读者我要写书评

暂无评论

Off-Policy Risk-Sensitive reinforcement learning-Based Constrained Robust Optimal Control

引用

ieee TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS 2023年第4期53卷 2478-2491页

作者： Li, Cong Liu, Qingchen Zhou, Zhehua Buss, Martin Liu, Fangzhou Tech Univ Munich Chair Automat Control Engn D-80333 Munich Germany Univ Sci & Technol China Dept Automat Hefei 230027 Peoples R China Harbin Inst Technol Res Ctr Intelligent Control & Syst Harbin 150001 Peoples R China

This article proposes an off-policy risk-sensitive reinforcement learning (RL)-based control framework to jointly optimize the task performance and constraint satisfaction in a disturbed environment. The risk-aware value function, constructed using the pseudo control and risk-sensitive input and state penalty terms, is introduced to convert the original constrained robust stabilization problem into an equivalent unconstrained optimal control problem. Then, an off-policy RL algorithm is developed to learn the approximate solution to the risk-aware value function. During the learning process, the associated approximate optimal control policy is able to satisfy both input and state constraints under disturbances. By replaying experience data to the off-policy weight update law of the critic neural network, the weight convergence is guaranteed. Moreover, online and offline algorithms are developed to serve as principled ways to record informative experience data to achieve a sufficient excitation required for the weight convergence. The proofs of system stability and weight convergence are provided. The Simulation results reveal the validity of the proposed control framework.

关键词： adaptive dynamic programming (ADP) input saturation off-policy risk-sensitive reinforcement learning (RL) robust control state constraint

来源：评论

学校读者我要写书评

暂无评论

Event-Triggered Local Control for Nonlinear Interconnected Systems Through Particle Swarm Optimization-Based adaptive dynamic programming

引用

ieee TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS 2023年第12期53卷 7342-7353页

作者： Zhao, Bo Shi, Guang Liu, Derong Beijing Normal Univ Sch Syst Sci Beijing 100875 Peoples R China Chongqing Univ Posts & Telecommun Key Lab Ind Internet Things & Networked Control Minist Educ Chongqing 400065 Peoples R China Coordinat Ctr China Dept Engn Natl Comp Network Emergency Response Tech Team Beijing 100029 Peoples R China Southern Univ Sci & Technol Sch Syst Design & Intelligent Mfg Shenzhen 518055 Peoples R China Univ Illinois Dept Elect & Comp Engn Chicago IL 60607 USA

This article investigates local control problems for nonlinear interconnected systems by using adaptive dynamic programming (ADP) with particle swarm optimization (PSO). Through constructing a proper local value function, a local critic neural network, whose weight vector is tuned via the PSO algorithm, is employed to solve the local Hamilton-Jacobi-Bellman equation. By introducing the event-triggering mechanism, the sampling time instants of each interconnected subsystem are determined by establishing a proper event-triggering condition. Then, the ADP-based event-triggered local control policy can be derived indirectly to ensure the closed-loop nonlinear interconnected system to be asymptotically stable through the Lyapunov stability analysis. The positive lower bound on the minimal intersampling instant for each interconnected subsystem is provided to exclude the Zeno behavior. Simulation results of a practical system and a numerical example demonstrate the effectiveness of the present event-triggered local control scheme.

关键词： adaptive dynamic programming (ADP) event-triggering mechanism local control nonlinear interconnected systems optimal control reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Approximate dynamic programming Solutions of Multi-Agent Graphical Games Using Actor-Critic Network Structures

Approximate Dynamic Programming Solutions of Multi-Agent Gra...

引用

International Joint Conference on Neural Networks (IJCNN)

作者： Abouheaf, Mohammed I. Lewis, Frank L. Univ Texas Arlington Res Inst Ft Worth TX 76118 USA

ISBN: (纸本)9781467361293;9781467361286

This paper studies a new class of multi-agent discrete-time dynamical graphical games, where interactions between agents are restricted by a communication graph structure. The paper brings together discrete Hamiltonian mechanics, optimal control theory, cooperative control, game theory, reinforcement learning, and neural network structures to solve the multi-agent dynamical graphical games. Graphical game Bellman equations are derived and shown to be equivalent to certain graphical game Hamilton Jacobi Bellman equations developed herein. reinforcement learning techniques are used to solve these dynamical graphical games. Heuristic dynamic programming and Dual Heuristic programming, are extended to solve the graphical games using only neighborhood information. Online adaptive learning structure is implemented using actor-critic networks to solve these graphical games.

关键词： discrete time systems dynamic programming game theory graph theory learning (artificial intelligence) multi-robot systems optimal control

来源：评论

学校读者我要写书评

暂无评论

Evaluation of policy gradient methods and variants on the cart-pole benchmark

Evaluation of policy gradient methods and variants on the ca...

引用

ieee International symposium on Approximate dynamic programming and reinforcement learning

作者： Riedmiller, Martin Peters, Jan Schaal, Stefan Univ Osnabruck Neuroinformat Grp D-4500 Osnabruck Germany Univ Southern Calif Computat Learning & Motor Control Los Angeles CA 90007 USA

ISBN: (纸本)9781424407064

In this paper, we evaluate different versions from the three main kinds of model-free policy gradient methods, i.e., finite difference gradients, 'vanilla' policy gradients and natural policy gradients. Each of these methods is first presented in its simple form and subsequently refined and optimized. By carrying out numerous experiments on the cart pole regulator benchmark we aim to provide a useful baseline for future research on parameterized policy search algorithms. Portable C++ code is provided for both plant and algorithms;thus, the results in this paper can be reevaluated, reused and new algorithms can be inserted with ease.

关键词： Gradient methods

来源：评论

学校读者我要写书评

暂无评论

A Hybrid Controller for Musculoskeletal Robots Targeting Lifting Tasks in Industrial Metaverse

引用

ieee TRANSACTIONS ON CYBERNETICS 2024年第5期54卷 2708-2719页

作者： Qin, Shijie Li, Houcheng Cheng, Long Chinese Acad Sci Inst Automat State Key Lab Multimodal Artificial Intelligence S Beijing 100190 Peoples R China Univ Chinese Acad Sci Sch Artificial Intelligence Beijing 100049 Peoples R China

In manufacturing, musculoskeletal robots have gained more attention with the potential advantages of flexibility, robustness, and adaptability over conventional serial-link rigid robots. Focusing on the fundamental lifting tasks, a hybrid controller is proposed to overcome control challenges of such robots for widely applications in industry. The metaverse technology offers an available simulated-reality-based platform to verify the proposed method. The hybrid controller contains two main parts. A muscle-synergy-based radial basis function (RBF) network is proposed as the feedforward controller, which is able to characterize the phasic and the tonic muscle synergies simultaneously. The adaptive dynamic programming (ADP) is applied as the feedback controller to address the optimal control problem. The actor-critic structure is applied in the ADP-based controller, where the critic network is trained to approximate the optimal performance index and the actor network is trained to compute the optimal muscle excitations. Furthermore, the convergence and stability of the ADP algorithm are also analyzed. Finally, experiments have been designed to verify the effectiveness of this hybrid controller on an upper limb musculoskeletal system, and the comparisons with other controllers are also illustrated. The results show that the proposed controller can obtain a satisfactory performance for lifting tasks.

关键词： adaptive dynamic programming (ADP) brain-inspired method muscle synergy musculoskeletal system reinforcement learning (RL) tracking control

来源：评论

学校读者我要写书评

暂无评论

adaptive Online Distributed Optimal Control of Very-Large-Scale Robotic Systems

引用

ieee TRANSACTIONS ON CONTROL OF NETWORK SYSTEMS 2021年第2期8卷 678-689页

作者： Zhu, Pingping Liu, Chang Ferrari, Silvia Marshall Univ Dept Comp Sci & Elect Engn Huntington WV 25705 USA Cornell Univ Sibley Sch Mech & Aerosp Engn Ithaca NY 14853 USA

Autonomous systems comprised of many cooperative agents have the potential for enabling long-duration tasks and data collection critical to the understanding of a wide range of phenomena in spatially and temporally variable environments. The adaptive distributed optimal control approach presented in this article extends online approximate dynamic programming to very-large-scale robotics (VLSR) systems that must operate and adapt to highly uncertain and variable environments. Optimal mass transport theory is used to show that, in the Wasserstein-Gaussian mixture model space, the VLSR system's cost to go can be represented by a value functional of the robot distribution and dynamic environmental maps. The approach is demonstrated on a cooperative path planning problem in which knowledge of the obstacles in the environment changes incrementally over time based on in situ measurements. Numerical simulations show that the proposed approach significantly outperforms existing methods by finding an approximately optimal solution that avoids obstacles and meets a desired final robot distribution using minimum energy.

关键词： adaptive cooperative environmental adaptation multiagent reinforcement learning (MARL) optimal control path planning very-large-scale robotic (VLSR)

来源：评论

学校读者我要写书评

暂无评论

learning to Regulate Rolling Ball Motion

Learning to Regulate Rolling Ball Motion

引用

ieee symposium Series on Computational Intelligence (ieee SSCI)

作者： Jha, Devesh K. Yerazunis, William Nikovski, Daniel Farahmand, Amir-massoud Mitsubishi Elect Res Labs Cambridge MA 02139 USA

ISBN: (纸本)9781538627266

In this paper, we present a problem of regulating the motion of a rolling ball in a one-dimensional space in the presence of non-linear effects of friction and contact. The regulation problem is solved using a model-based reinforcement learning technique. A Gaussian process model is learned to make predictions on the motion of the ball and then, the predictive model is used to solve for the control policy using dynamic programming by estimating the value functions. Several results are shown to demonstrate the simple, yet interesting motion dynamics for the ball. Our hope is that the proposed system will serve as a simple benchmark system for reinforcement and robot learning.

关键词： reinforcement learning Gaussian Process Position Regulation

来源：评论

学校读者我要写书评

暂无评论

learning with binary-valued utility using derivative adaptive critic methods

Learning with binary-valued utility using derivative adaptiv...

引用

2004 ieee International Joint Conference on Neural Networks - Proceedings

作者： Matzner, Shari A. Shannon, Thaddeus T. Lendaris, George G. NW Compl. Intelligence Lab Systems Science Ph.D. Program Portland State University P.O. Box 751 Portland OR 97207 United States

ISBN: (纸本)0780383591

adaptive Critic methods for reinforcement learning are known to provide consistent solutions to optimal control problems, and are also considered plausible models for cognitive learning processes. This paper discusses binary reinforcement in the context of three adaptive Critic methods: Heuristic dynamic programming (HDP), Dual Heuristic programming (DHP), and Globalized Dual Heuristic programming (GDHP). Binary reinforcement arises when the qualitative measure of success is simply "pass" or "fail". We implement binary reinforcement with adaptive Critic methods for the pole-cart benchmark problem. Results demonstrate two qualitatively dissimilar classes of controllers: those that replicate the system stabilization achieved with quadratic utility, and those that merely succeed at not dropping the pole. It is found that the GDHP method is effective for learning an approximately optimal solution, with results comparable to those obtained via DHP that uses a more informative, quadratic utility function.

关键词： learning systems

来源：评论

学校读者我要写书评

暂无评论

A Low-Power Circuit for adaptive dynamic programming 31

A Low-Power Circuit for Adaptive Dynamic Programming

引用

31st International Conference on VLSI Design / 17th International Conference on Embedded Systems (VLSID & ES))

作者： Zheng, Nan Mazumder, Pinaki Univ Michigan Elect Engn & Comp Sci Dept Ann Arbor MI 48109 USA

ISBN: (纸本)9781538636923

This paper presents a low-power CMOS design for accelerating an adaptive dynamic programming algorithm, called action-dependent heuristic dynamic programming, which is widely employed in many real-life control problems. The objective of this work is to solve Bellman equation approximately using efficient hardware in order to generate near-optimal real-time control policies for many control applications. The hardware exploits the data-level parallelism exists in both inference and learning of neural networks in order to improve throughput as well as to provide good scalability. The circuit is realized in a 65-nm technology. It is shown with simulations that the design is two orders of magnitude faster than a software running on a general-purpose processor thanks to the parallelization of the algorithm and the reduction in unnecessary control overheads. Performance of the CMOS circuit is benchmarked with two popular control tasks. Successful learning can be achieved with a power consumption of 28 mW.

关键词： reinforcement learning adaptive dynamic programming neural networks low-power CMOS circuit

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：