检索结果-内蒙古大学图书馆

4th ieee International symposium on adaptive dynamic programming and reinforcement learning (ADPRL)

作者： Cai, Yifan Yang, Simon X. Xu, Xin Univ Guelph Sch Engn Guelph ON N1G 2W1 Canada Natl Univ Def Technol Coll Mechatron & Automat Changsha 410073 Hunan Peoples R China

ISBN: (纸本)9781467359252

Effective cooperation of multi-robots in unknown environments is essential in many robotic applications, such as environment exploration and target searching. In this paper, a combined hierarchical reinforcement learning approach, together with a designed cooperation strategy, is proposed for the real-time cooperation of multi-robots in completely unknown environments. Unlike other algorithms that need an explicit environment model or select parameters by trial and error, the proposed cooperation method obtains all the required parameters automatically through learning. By integrating segmental options with the traditional MAXQ algorithm, the cooperation hierarchy is built. In new tasks, the designed cooperation method can control the multi-robot system to complete the task effectively. The simulation results demonstrate that the proposed scheme is able to effectively and efficiently lead a team of robots to cooperatively accomplish target searching tasks in completely unknown environments.

关键词： Hierarchical reinforcement learning multi-robot cooperation complex unknown environment target searching

来源：评论

学校读者我要写书评

暂无评论

adaptive critic designs

引用

ieee TRANSACTIONS ON NEURAL NETWORKS 1997年第5期8卷 997-1007页

作者： Prokhorov, DV Wunsch, DC Laboratory of Chromatography DEPg.Fac.Quimica Universidad Nacional Autonoma de Mexico Circuito interior Cd Universitaria/CP 04510 Mexico D.F.Mexico

We discuss a variety of adaptive critic designs (ACD's) for neurocontrol. These are suitable for learning in noisy, nonlinear, and nonstationary environments. They have common roots as generalizations of dynamic programming for neural reinforcement learning approaches. Our discussion of these origins leads to an explanation of three design families: Heuristic dynamic programming (HDP), dual heuristic programming (DHP), and globalized dual heuristic programming (GDHP). The main emphasis is on DHP and GDHP as advanced ACD's. We suggest two new modifications of the original GDHP design that are currently the only working implementations of GDHP. They promise to be useful for many engineering applications in the areas of optimization and optimal control. Based on one of these modifications, we present a unified approach to all ACD's. This leads to a generalized training procedure for ACD's.

关键词： adaptive critic design (ACD) backpropagation control DHP dynamic programming GDHP HDP heuristic dynamic programming neural network neurocontrol reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Delayed Insertion and Rule Effect Moderation of Domain Knowledge for reinforcement learning

Delayed Insertion and Rule Effect Moderation of Domain Knowl...

引用

4th ieee International symposium on adaptive dynamic programming and reinforcement learning (ADPRL)

作者： Teng, Teck-Hou Tan, Ah-Hwee Nanyang Technol Univ Sch Comp Engn Ctr Computat Intelligence Singapore Singapore Nanyang Technol Univ Sch Comp Engn Singapore Singapore

ISBN: (纸本)9781467359252

Though not a fundamental pre-requisite to efficient machine learning, insertion of domain knowledge into adaptive virtual agent is nonetheless known to improve learning efficiency and reduce model complexity. Conventionally, domain knowledge is inserted prior to learning. Despite being effective, such approach may not always be feasible. Firstly, the effect of domain knowledge is assumed and can be inaccurate. Also, domain knowledge may not be available prior to learning. In addition, the insertion of domain knowledge can frame learning and hamper the discovery of more effective knowledge. Therefore, this work advances the use of domain knowledge by proposing to delay the insertion and moderate the effect of domain knowledge to reduce the framing effect while still benefiting from the use of domain knowledge. Using a non-trivial pursuit-evasion problem domain, experiments are first conducted to illustrate the impact of domain knowledge with different degrees of truth. The next set of experiments illustrates how delayed insertion of such domain knowledge can impact learning. The final set of experiments is conducted to illustrate how delaying the insertion and moderating the assumed effect of domain knowledge can ensure the robustness and versatility of reinforcement learning.

关键词： domain knowledge learning (artificial intelligence) final sets learning Insertion Mutation

来源：评论

学校读者我要写书评

暂无评论

Bias-Corrected Q-learning to Control Max-Operator Bias in Q-learning

Bias-Corrected Q-Learning to Control Max-Operator Bias in Q-...

引用

4th ieee International symposium on adaptive dynamic programming and reinforcement learning (ADPRL)

作者： Lee, Donghun Defourny, Boris Powell, Warren B. Princeton Univ Dept Comp Sci Princeton NJ 08540 USA Princeton Univ Dept Operat Res & Financial Engn Princeton NJ 08540 USA

ISBN: (纸本)9781467359252

We identify a class of stochastic control problems with highly random rewards and high discount factor which induce high levels of statistical error in the estimated action-value function. This produces significant levels of max-operator bias in Q-learning, which can induce the algorithm to diverge for millions of iterations. We present a bias-corrected Q-learning algorithm with asymptotically unbiased resistance against the max-operator bias, and show that the algorithm asymptotically converges to the optimal policy, as Q-learning does. We show experimentally that bias-corrected Q-learning performs well in a domain with highly random rewards where Q-learning and other related algorithms suffer from the max-operator bias.

关键词： Bias Bias Bias Statistical Biases bias(electronics) statistical error Reward optimal strategy iterative methods algorithms asymptotic

来源：评论

学校读者我要写书评

暂无评论

Scalarized Multi-Objective reinforcement learning: Novel Design Techniques

Scalarized Multi-Objective Reinforcement Learning: Novel Des...

引用

4th ieee International symposium on adaptive dynamic programming and reinforcement learning (ADPRL)

作者： Van Moffaert, Kristof Drugan, Madalina M. Nowe, Ann Vrije Univ Brussel Dept Comp Sci B-1050 Brussels Belgium

ISBN: (纸本)9781467359252

In multi-objective problems, it is key to find compromising solutions that balance different objectives. The linear scalarization function is often utilized to translate the multi-objective nature of a problem into a standard, single-objective problem. Generally, it is noted that such as linear combination can only find solutions in convex areas of the Pareto front, therefore making the method inapplicable in situations where the shape of the front is not known beforehand, as is often the case. We propose a non-linear scalarization function, called the Chebyshev scalarization function, as a basis for action selection strategies in multi-objective reinforcement learning. The Chebyshev scalarization method overcomes the flaws of the linear scalarization function as it can (i) discover Pareto optimal solutions regardless of the shape of the front, i. e. convex as well as non-convex, (ii) obtain a better spread amongst the set of Pareto optimal solutions and (iii) is not particularly dependent on the actual weights used.

关键词： design techniques learning (artificial intelligence) optimal solution Convexity flaws learning Linear Combination Solutions

来源：评论

学校读者我要写书评

暂无评论

adaptive Optimal Control of CVCF Inverters With Uncertain Load: An adaptive dynamic programming Approach

引用

ieee ACCESS 2021年 9卷 89276-89286页

作者： Wang, Zhongyang Yu, Yunjun Fuzhou Inst Technol Sch Appl Sci & Engn Fuzhou 350506 Peoples R China Nanchang Univ Sch Informat Engn Nanchang 330031 Jiangxi Peoples R China Nanchang Univ AI Inst Nanchang 330031 Jiangxi Peoples R China

This paper proposed a data-driven adaptive optimal control approach for CVCF (constant voltage, constant frequency) inverter based on reinforcement learning and adaptive dynamic programming (ADP). Different from existing literature, the load is treated as a dynamic uncertainty and a robust optimal state-feedback controller is proposed. The stability of the inverter-load system has been strictly analyzed. In order to obtain accurate output current differential signal, this paper designs a tracking differentiator. It is ensured that the tracking error asymptotically converges to zero through the proposed output-feedback controllers. A standard proportional integral controller and linear active disturbance rejection control strategy are also designed for the purpose of comparison. The simulation results show that the proposed controller has inherent robustness and does not require retuning with different applications.

关键词： Inverters Voltage control Mathematical model reinforcement learning Optimal control Load modeling Licenses adaptive optimal control CVCF inverter reinforcement learning adaptive dynamic programming

来源：评论

学校读者我要写书评

暂无评论

A Study on the Efficiency of learning a Robot Controller in Various Environments

A Study on the Efficiency of Learning a Robot Controller in ...

引用

4th ieee International symposium on adaptive dynamic programming and reinforcement learning (ADPRL)

作者： Soga, Sachiko Kobayashi, Ichiro Ochanomizu Univ Grad Sch Humanities & Sci Bunkyo Ku Tokyo 1128610 Japan

ISBN: (纸本)9781467359252

In the case that a robot controller is trained by means of evolutionary computation, the robot will be able to behave sufficiently in the environment where the robot has been trained. However, if the robot is put in an environment which is more complex than a training environment, it cannot behave sufficiently and is required to be trained again so as it fits to the complex environment. Based on this fact, we build a training environment for a robot controller with the partial components of a more complex environment than the training environment and aim to obtain a controller which makes a robot be able to act in the complex environment by only training the controller at a simpler environment. We clarify a way of building a training environment which functions effectively for training a robot controller and discuss how much training is necessary in the training environment for a robot to be able to behave under a more complex environment.

关键词： robot controller Environments Robot Functional training Controllers

来源：评论

学校读者我要写书评

暂无评论

reinforcement learning to Train Ms. Pac-Man Using Higher-order Action-relative Inputs

Reinforcement Learning to Train Ms. Pac-Man Using Higher-ord...

引用

4th ieee International symposium on adaptive dynamic programming and reinforcement learning (ADPRL)

作者： Bom, Luuk Henken, Ruud Wiering, Marco Univ Groningen Inst Artificial Intelligence & Cognit Engn Fac Math & Nat Sci NL-9700 AB Groningen Netherlands

ISBN: (纸本)9781467359252

reinforcement learning algorithms enable an agent to optimize its behavior from interacting with a specific environment. Although some very successful applications of reinforcement learning algorithms have been developed, it is still an open research question how to scale up to large dynamic environments. In this paper we will study the use of reinforcement learning on the popular arcade video game Ms. Pac-Man. In order to let Ms. Pac-Man quickly learn, we designed particular smart feature extraction algorithms that produce higher-order inputs from the game-state. These inputs are then given to a neural network that is trained using Q-learning. We constructed higher-order features which are relative to the action of Ms. Pac-Man. These relative inputs are then given to a single neural network which sequentially propagates the action-relative inputs to obtain the different Q-values of different actions. The experimental results show that this approach allows the use of only 7 input units in the neural network, while still quickly obtaining very good playing behavior. Furthermore, the experiments show that our approach enables Ms. Pac-Man to successfully transfer its learned policy to a different maze on which it was not trained before.

关键词： learning (artificial intelligence) higher-order learning Neural network MILLISECOND

来源：评论

学校读者我要写书评

暂无评论

Optimistic Planning for Continuous-Action Deterministic Systems

Optimistic Planning for Continuous-Action Deterministic Syst...

引用

4th ieee International symposium on adaptive dynamic programming and reinforcement learning (ADPRL)

作者： Busoniu, Lucian Daniels, Alexander Munos, Remi Babuska, Robert Univ Lorraine CRAN UMR 7039 Nancy France CNRS CRAN UMR 7039 Nancy France Delft Univ Technol DCSC Delft Netherlands INRIA Lille Nord Europe Team SequeL Lille France

ISBN: (纸本)9781467359252

We consider the class of online planning algorithms for optimal control, which compared to dynamic programming are relatively unaffected by large state dimensionality. We introduce a novel planning algorithm called SOOP that works for deterministic systems with continuous states and actions. SOOP is the first method to explore the true solution space, consisting of infinite sequences of continuous actions, without requiring knowledge about the smoothness of the system. SOOP can be used parameter-free at the cost of more model calls, but we also propose a more practical variant tuned by a parameter a, which balances finer discretization with longer planning horizons. Experiments on three problems show SOOP reliably ranks among the best algorithms, fully dominating competing methods when the problem requires both long horizons and fine discretization.

关键词： Solution space dynamic programming Smoothness deterministic Optimal control continuous action Discretization Horizon Planning

来源：评论

学校读者我要写书评

暂无评论

The Second Order Temporal Difference Error for Sarsa(λ)

The Second Order Temporal Difference Error for Sarsa(λ)

引用

4th ieee International symposium on adaptive dynamic programming and reinforcement learning (ADPRL)

作者： Fu, Qiming Liu, Quan Xiao, Fei Chen, Guixin Soochow Univ Dept Comp Sci & Technol Suzhou Peoples R China

ISBN: (纸本)9781467359252

Traditional reinforcement learning algorithms, such as Q-learning, Q(lambda), Sarsa, and Sarsa(lambda), update the action value function using temporal difference (TD) error, which is computed by the last action value function. From the perspective of the TD error, and with respect to the problems of low efficiency and slow convergence of the traditional Sarsa(lambda) algorithm, this paper defines the nth order TD Error, applies it in the traditional Sarsa(lambda) algorithm, and develops a fast Sarsa(lambda) algorithm based on the 2nd order TD Error. The algorithm adjusts the Q value with the second-order TD Error and broadcasts the TD Error into the whole state-action space, which speeds up the convergence of the algorithm. This paper also analyzes the convergence rate, and under the condition of one-step update, the results show that the number of iteration depends primarily on gamma, epsilon. Finally, using the proposed algorithm on the traditional reinforcement learning problems, the results show that the algorithm has both a faster convergence rate and better convergence performance.

关键词： reinforcement learning Markov Decision Process Second Order TD Error Eligibility Trace Sarsa(lambda) Algorithm

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：