检索结果-内蒙古大学图书馆

multi-task reinforcement learning Based on Parallel Recombination Networks

IEEE ACCESS 2025年 13卷 80113-80122页

作者： Liu, Manlu Zhang, Qingbo Qian, Weimin Southwest Univ Sci & Technol Sch Informat Engn Mianyang 621010 Peoples R China Robot Technol Used Special Environm Key Lab Sichua Mianyang 621010 Peoples R China

multi-task reinforcement learning is a key current trend in the field of reinforcement learning. It can accomplish multiple tasks using a single network, which is superior to single-task learning in integrating information from different tasks. However, uncertainty remains on the issue of how to effectively share parameters across tasks in the network. To address this problem, this paper proposes a 'soft parallel recombination network' approach, which can share task information across network layers without being limited to between adjacent layers, thus enhancing the information sharing capability of the network. Specifically, the types of multi-task learning in this paper include various manipulator control tasks executed in the Meta-world environment, such as pick-and-place, push, and stacking. For optimal performance, a weight network is introduced which automatically determines the optimal path for each task and outputs the probability of each module being selected. The proposed method efficiently learns the relationships between tasks from parallel recombination networks and determines the optimal path for a task through a weight network. Further, the weight relationship between the current training samples and the current strategy is found, which improves the training efficiency in combination with the parallel recombination network. By combining the proposed 'Soft Parallel Recombination Network' method with the SAC algorithm (PRSAC) and validating it on the Meta-world multi-task training platform, the experimental results demonstrate that the proposed method significantly outperforms existing baseline algorithms in terms of sample efficiency and performance.

关键词： multitasking reinforcement learning Training Robots Manipulators Optimization Metaverse multi-task reinforcement learning meta-world parallel recombination network robotic manipulation task

来源：评论

学校读者我要写书评

暂无评论

multi-task reinforcement learning With Attention-Based Mixture of Experts

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2023年第6期8卷 3811-3818页

作者： Cheng, Guangran Dong, Lu Cai, Wenzhe Sun, Changyin Southeast Univ Sch Automat Nanjing 210096 Peoples R China Southeast Univ Sch Cyber Sci & Engn Nanjing 211189 Peoples R China

multi-task learning is an important problem in reinforcement learning. Training multiple tasks together brings benefits from the shared useful information across different tasks and often achieves higher performance compared to single-task learning. However, it remains unclear how parameters in the network should be reused across tasks. Instead of naively sharing parameters across all tasks, we propose an attention-based mixture of experts multi-task reinforcement learning approach to learn a compositional policy for each task. The expert networks learn task-specific skills which specialize in different parts of multi-task representation space. To assemble the expert outputs, we propose an attention module to generate connections between tasks and experts to achieve the best performance automatically. The proposed approach can effectively learn task relationships from soft attention weights and alleviate mutual interference through independent expert networks. We use the proposed approach to improve both sample efficiency and overall performance over baseline algorithms in Meta-world, a benchmark for multi-task reinforcement learning containing 50 robotic manipulation tasks, and the multi-task MUJOCO environment, which contains four considerably different tasks with diverse state transition dynamics.

关键词： task analysis multitasking reinforcement learning Training Robots Neural networks Interference multi-task reinforcement learning mixture of experts attention Meta-World robotic manipulation task

来源：评论

学校读者我要写书评

暂无评论

multi-task reinforcement learning with Cost-based HTN Planning 5

Multi-Task Reinforcement Learning with Cost-based HTN Planni...

引用

5th International Conference on Computer Engineering and Application (ICCEA)

作者： Hu, Yuyong Zhuo, Hankz Hankui Sun Yat Sen Univ Sch Comp Sci & Engn Guangzhou Peoples R China

ISBN: (纸本)9798350386783;9798350386776

multi-task reinforcement learning (MT-RL) faces key challenges in accomplishing complex long-horizon tasks, particularly related to scarce rewards, inefficient sample usage, and low transferability. These challenges are exacerbated in real world scenarios where tasks can often be done by completing different intermediate subtasks, complicating intermediate reward allocation. To address those issues, we introduce a novel framework integrating a strategic planner, a pre-trained language module, and a reinforcement learning policy. This framework strategically decomposes complex tasks into observable sub-task lists using the planner, adapting the plan based on sub-task completion, while the incorporation of the pre-trained language module aids in the task list understanding. We evaluated our framework in a single-agent overcooked environment, chosen for its relevance in the long-horizon tasks. Our results demonstrate notable improvements in time efficiency and adaptability, showcasing the framework's potential to enhance MT-RL applications.

关键词： multi-task reinforcement learning Planning Hierarchical task Network Pre-trained Language Models

来源：评论

学校读者我要写书评

暂无评论

Finite-Time Complexity of Incremental Policy Gradient Methods for Solving multi-task reinforcement learning 6

Finite-Time Complexity of Incremental Policy Gradient Method...

引用

6th Annual learning for Dynamics and Control Conference

作者： Bai, Yitao Doan, Thinh T. Virginia Tech Bradley Dept Elect & Comp Engn Blacksburg VA 24061 USA

We consider a multi-task learning problem, where an agent is presented a number of N reinforcement learning tasks. To solve this problem, we are interested in studying the gradient approach, which iteratively updates an estimate of the optimal policy using the gradients of the value functions. The classic policy gradient method, however, may be expensive to implement in the multi-task settings as it requires access to the gradients of all the tasks at every iteration. To circumvent this issue, in this paper we propose to study an incremental policy gradient method, where the agent only uses the gradient of only one task at each iteration. Our main contribution is to provide theoretical results to characterize the performance of the proposed method. In particular, we show that incremental policy gradient methods converge to the optimal value of the multi-task reinforcement learning objectives at a sublinear rate O(1/root k), where k is the number of iterations. To illustrate its performance, we apply the proposed method to solve a simple multi-task variant of GridWorld problems, where an agent seeks to find an policy to navigate effectively in different environments.

关键词： multi-task reinforcement learning Markov Decision Processes Incremental Policy Gradient Methods

来源：评论

学校读者我要写书评

暂无评论

Modular Networks Prevent Catastrophic Interference in Model-Based multi-task reinforcement learning 7th

Modular Networks Prevent Catastrophic Interference in Model-...

引用

7th International Conference on Machine learning, Optimization, and Data Science (LOD) / 1st Symposium on Artificial Intelligence and Neuroscience (ACAIN)

作者： Schiewer, Robin Wiskott, Laurenz Ruhr Univ Bochum Inst Neural Computat Bochum Germany

ISBN: (纸本)9783030954703;9783030954697

In a multi-task reinforcement learning setting, the learner commonly benefits from training on multiple related tasks by exploiting similarities among them. At the same time, the trained agent is able to solve a wider range of different problems. While this effect is well documented for model-free multi-task methods, we demonstrate a detrimental effect when using a single learned dynamics model for multiple tasks. Thus, we address the fundamental question of whether model-based multi-task reinforcement learning benefits from shared dynamics models in a similar way model-free methods do from shared policy networks. Using a single dynamics model, we see clear evidence of task confusion and reduced performance. As a remedy, enforcing an internal structure for the learned dynamics model by training isolated sub-networks for each task notably improves performance while using the same amount of parameters. We illustrate our findings by comparing both methods on a simple gridworld and a more complex vizdoom multi-task experiment.

关键词： Model-based reinforcement learning multi-task reinforcement learning Latent space models Catastrophic interference task confusion

来源：评论

学校读者我要写书评

暂无评论

Scaling Up multi-task Robotic reinforcement learning 5

Scaling Up Multi-Task Robotic Reinforcement Learning

引用

5th Conference on Robot learning (CoRL)

作者： Kalashnikov, Dmitry Varley, Jake Chebotar, Yevgen Swanson, Benjamin Jonschkowski, Rico Finn, Chelsea Levine, Sergey Hausman, Karol Google Team Google Res Robot Mountain View CA 94043 USA

General-purpose robotic systems must master a large repertoire of diverse skills. While reinforcement learning provides a powerful framework for acquiring individual behaviors, the time needed to acquire each skill makes the prospect of a generalist robot trained with RL daunting. We study how a large-scale collective robotic learning system can acquire a repertoire of behaviors simultaneously, sharing exploration, experience, and representations across tasks. In this framework, new tasks can be continuously instantiated from previously learned tasks improving overall performance and capabilities of the system. To this end, we develop a scalable and intuitive framework for specifying new tasks through user-provided examples of desired outcomes, devise a multi-robot collective learning system for data collection that simultaneously collects experience for multiple tasks, and develop a scalable and generalizable multi-task deep reinforcement learning method, which we call MT-Opt. We demonstrate how MT-Opt can learn a wide range of skills, including semantic picking (i.e., picking an object from a particular category), placing into various fixtures (e.g., placing a food item onto a plate). We train and evaluate our system on a set of 12 real-world tasks with data collected from 7 robots, and demonstrate the performance of our system both in terms of its ability to generalize to structurally similar new tasks, and acquire distinct new tasks more quickly by leveraging past experience. We recommend viewing the videos at https://***/mt-opt/

关键词： multi-task reinforcement learning Robot learning

来源：评论

学校读者我要写书评

暂无评论

Particle swarm optimization based multi-task parallel reinforcement learning algorithm

引用

JOURNAL OF INTELLIGENT & FUZZY SYSTEMS 2019年第6期37卷 8567-8575页

作者： Duan Junhua Zhu Yi-an Zhong Dong Zhang Lixiang Zhang Lin Northwestern Polytech Univ Sch Comp 127 West Youyi Rd Xian 710072 Shaanxi Peoples R China

Transfer learning has been identified as conducive to improving the speed of machine learning in many areas. In multi-task reinforcement learning, transfer learning can assist the transfer of experiences between different tasks. The research conducted in this article is focused on two aspects. On the one hand, multi-task parallel transfer learning can improve the learning speed of parallel learning tasks. On the other hand, the learning of the current optimal experience can help the target point rewards to be transmitted to the starting point. The value of this self-learning can also accelerate the convergence speed of the reinforcement learning. According to the research into these two aspects, this paper uses the idea of particle swarm optimization (PSO) to conduct self-learning and interactive learning in multi-task parallel learning. In this paper, a new multi-task learning algorithm named PSO-MTPRL (multi-task Parallel reinforcement learning based on PSO) is proposed. Based on the idea of PSO algorithm, the Boltzmann strategy, Self-learning Process (SLP) and Interactive learning Process (ILP) are selected probabilistically. Based on the characteristic exhibited by reinforcement learning, segmented learning model is recommended. In the early learning stages, the complete Boltzmann exploration strategy is applied, and B-SLP-ILP (Boltzmann-SLP- ILP) learning procedure is conducted exclusively in the middle stage of the learning. In the late learning stages, Boltzmann exploration is involved again. The segmented learning model can help ensure the balance of the exploration and exploitation, in addition to ensuring that all tasks convergence.

关键词： multi-task reinforcement learning parallel reinforcement learning particle swarm optimization transfer learning

来源：评论

学校读者我要写书评

暂无评论

Meta-World: A Benchmark and Evaluation for multi-task and Meta reinforcement learning 3

Meta-World: A Benchmark and Evaluation for Multi-Task and Me...

引用

3rd Conference on Robot learning (CoRL)

作者： Yu, Tianhe Quillen, Deirdre He, Zhanpeng Julian, Ryan Hausman, Karol Finn, Chelsea Levine, Sergey Stanford Univ Stanford CA 94305 USA Univ Calif Berkeley Berkeley CA USA Columbia Univ New York NY 10027 USA Univ Southern Calif Los Angeles CA 90007 USA Google Robot Mountain View CA 94043 USA

Meta-reinforcement learning algorithms can enable robots to acquire new skills much more quickly, by leveraging prior experience to learn how to learn. However, much of the current research on meta-reinforcement learning focuses on task distributions that are very narrow. For example, a commonly used meta-reinforcement learning benchmark uses different running velocities for a simulated robot as different tasks. When policies are meta-trained on such narrow task distributions, they cannot possibly generalize to more quickly acquire entirely new tasks. Therefore, if the aim of these methods is enable faster acquisition of entirely new behaviors, we must evaluate them on task distributions that are sufficiently broad to enable generalization to new behaviors. In this paper, we propose an open-source simulated benchmark for meta-reinforcement learning and multi-task learning consisting of 50 distinct robotic manipulation tasks. Our aim is to make it possible to develop algorithms that generalize to accelerate the acquisition of entirely new, held-out tasks. We evaluate 6 state-of-the-art meta-reinforcement learning and multi-task learning algorithms on these tasks. Surprisingly, while each task and its variations (e.g., with different object positions) can be learned with reasonable success, these algorithms struggle to learn with multiple tasks at the same time, even with as few as ten distinct training tasks. Our analysis and open-source environments pave the way for future research in multi-task learning and meta-learning that can enable meaningful generalization, thereby unlocking the full potential of these methods.

关键词： meta-learning multi-task reinforcement learning benchmarks

来源：评论

学校读者我要写书评

暂无评论

learning potential functions and their representations for multi-task reinforcement learning

引用

AUTONOMOUS AGENTS AND multi-AGENT SYSTEMS 2014年第4期28卷 637-681页

作者： Snel, Matthijs Whiteson, Shimon Univ Amsterdam Intelligent Syst Lab Amsterdam Amsterdam Netherlands

In multi-task learning, there are roughly two approaches to discovering representations. The first is to discover task relevant representations, i.e., those that compactly represent solutions to particular tasks. The second is to discover domain relevant representations, i.e., those that compactly represent knowledge that remains invariant across many tasks. In this article, we propose a new approach to multi-task learning that captures domain-relevant knowledge by learning potential-based shaping functions, which augment a task's reward function with artificial rewards. We address two key issues that arise when deriving potential functions. The first is what kind of target function the potential function should approximate;we propose three such targets and show empirically that which one is best depends critically on the domain and learning parameters. The second issue is the representation for the potential function. This article introduces the notion of -relevance, the expected relevance of a representation on a sample sequence of tasks, and argues that this is a unifying definition of relevance of which both task and domain relevance are special cases. We prove formally that, under certain assumptions, -relevance converges monotonically to a fixed point as increases, and use this property to derive Feature Selection Through Extrapolation of k-relevance (FS-TEK), a novel feature-selection algorithm. We demonstrate empirically the benefit of FS-TEK on artificial domains.

关键词： multi-task reinforcement learning Feature selection Abstraction Potential-based shaping Transfer learning

来源：评论

学校读者我要写书评

暂无评论

Coordinating multi-Energy Microgrids for Integrated Energy System Resilience: A multi-task learning Approach

引用

IEEE TRANSACTIONS ON SUSTAINABLE ENERGY 2024年第2期15卷 920-937页

作者： Wang, Yi Qiu, Dawei Sun, Xiaotian Bie, Zhaohong Strbac, Goran Imperial Coll London Dept Elect & Elect Engn London SW7 2AZ England Xi An Jiao Tong Univ Sch Elect Engn Xian 710049 Peoples R China

High-impact and low-probability events have occurred more frequently than before, which can seriously damage energy supply infrastructures. As localized small energy systems, multi-energy microgrids (MEMGs) can provide a viable solution for the system-wise load restoration of integrated energy systems (IESs), due to their enhanced flexibility and controllability. However, existing literature tends to realize MEMGs as corrective response rather than load restoration resource after extreme events, which cannot fully exploit the benefits of multi-MEMGs on IES resilience. This article introduces a decentralized operating paradigm for the real-time coordination of local multi-MEMGs towards system-wise IES load restoration, while a novel topology-aware multi-task reinforcement learning method with soft modularization is proposed to solve it. The multi-task learning framework enables MEMGs to simultaneously learn scheduling decisions across different network topologies and better adapt to unanticipated contingencies. Additionally, to avoid insecure MEMG operations, a physics-informed safety layer is embedded on top of the multi-task learning framework for action corrections. Case studies have been conducted on two IESs (33-bus power, 20-node gas, and 20-node heat network as well as 69-bus power, 40-node gas, and 62-node heat network) to evaluate the effectiveness of the proposed method in enabling effective coordination among multi-MEMGs towards system-wise IES load restoration.

关键词： Integrated energy systems multi-energy microgrids multi-task reinforcement learning resilience safety layer

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：