检索结果-内蒙古大学图书馆

8th Conference on Robot learning, CoRL 2024

作者： Jia, Xiaogang Wang, Qian Donat, Atalay Xing, Bowen Li, Ge Zhou, Hongyi Celik, Onur Blessing, Denis Lioutikov, Rudolf Neumann, Gerhard Autonomous Learning Robots Karlsruhe Institute of Technology Germany Intuitive Robots Lab Karlsruhe Institute of Technology Germany

This work presents Mamba Imitation learning (MaIL), a novel imitation learning (IL) architecture that provides an alternative to state-of-the-art (SoTA) Transformer-based policies. MaIL leverages Mamba, a state-space model designed to selectively focus on key features of the data. While Transformers are highly effective in data-rich environments due to their dense attention mechanisms, they can struggle with smaller datasets, often leading to overfitting or suboptimal representation learning. In contrast, Mamba's architecture enhances representation learning efficiency by focusing on key features and reducing model complexity. This approach mitigates overfitting and enhances generalization, even when working with limited data. Extensive evaluations on the LIBERO benchmark demonstrate that MaIL consistently outperforms Transformers on all LIBERO tasks with limited data and matches their performance when the full dataset is available. Additionally, MaIL's effectiveness is validated through its superior performance in three real robot experiments. Our code is available at https://***/ALRhub/MaIL. © 2024 Proceedings of Machine learning Research.

关键词： Contrastive learning

来源：评论

学校读者我要写书评

暂无评论

TOWARDS DIVERSE BEHAVIORS: A BENCHMARK FOR IMITATION learning WITH HUMAN DEMONSTRATIONS 12

TOWARDS DIVERSE BEHAVIORS: A BENCHMARK FOR IMITATION LEARNIN...

引用

12th International Conference on learning Representations, ICLR 2024

作者： Jia, Xiaogang Blessing, Denis Jiang, Xinkai Reuss, Moritz Donat, Atalay Lioutikov, Rudolf Neumann, Gerhard Autonomous Learning Robots Karlsruhe Institute of Technology Germany Intuitive Robots Lab Karlsruhe Institute of Technology Germany

Imitation learning with human data has demonstrated remarkable success in teaching robots in a wide range of skills. However, the inherent diversity in human behavior leads to the emergence of multi-modal data distributions, thereby presenting a formidable challenge for existing imitation learning algorithms. Quantifying a model's capacity to capture and replicate this diversity effectively is still an open problem. In this work, we introduce simulation benchmark environments and the corresponding Datasets with Diverse human Demonstrations for Imitation learning (D3IL), designed explicitly to evaluate a model's ability to learn multi-modal behavior. Our environments are designed to involve multiple sub-tasks that need to be solved, consider manipulation of multiple objects which increases the diversity of the behavior and can only be solved by policies that rely on closed loop sensory feedback. Other available datasets are missing at least one of these challenging properties. To address the challenge of diversity quantification, we introduce tractable metrics that provide valuable insights into a model's ability to acquire and reproduce diverse behaviors. These metrics offer a practical means to assess the robustness and versatility of imitation learning algorithms. Furthermore, we conduct a thorough evaluation of state-of-the-art methods on the proposed task suite. This evaluation serves as a benchmark for assessing their capability to learn diverse behaviors. Our findings shed light on the effectiveness of these methods in tackling the intricate problem of capturing and generalizing multi-modal human behaviors, offering a valuable reference for the design of future imitation learning algorithms. Project page: https://***/d3il-website/. © 2024 12th International Conference on learning Representations, ICLR 2024. All rights reserved.

关键词： learning algorithms

来源：评论

学校读者我要写书评

暂无评论

Variational Distillation of Diffusion Policies into Mixture of Experts 38

Variational Distillation of Diffusion Policies into Mixture ...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Zhou, Hongyi Blessing, Denis Li, Ge Celik, Onur Jia, Xiaogang Neumann, Gerhard Lioutikov, Rudolf Intuitive Robots Lab Karlsruhe Institute of Technology Germany Autonomous Learning Robots Karlsruhe Institute of Technology Germany FZI Research Center for Information Technology Germany

This work introduces Variational Diffusion Distillation (VDD), a novel method that distills denoising diffusion policies into Mixtures of Experts (MoE) through variational inference. Diffusion Models are the current state-of-the-art in generative modeling due to their exceptional ability to accurately learn and represent complex, multi-modal distributions. This ability allows Diffusion Models to replicate the inherent diversity in human behavior, making them the preferred models in behavior learning such as learning from Human Demonstrations (LfD). However, diffusion models come with some drawbacks, including the intractability of likelihoods and long inference times due to their iterative sampling process. The inference times, in particular, pose a significant challenge to real-time applications such as robot control. In contrast, MoEs effectively address the aforementioned issues while retaining the ability to represent complex distributions but are notoriously difficult to train. VDD is the first method that distills pre-trained diffusion models into MoE models, and hence, combines the expressiveness of Diffusion Models with the benefits of Mixture Models. Specifically, VDD leverages a decompositional upper bound of the variational objective that allows the training of each expert separately, resulting in a robust optimization scheme for MoEs. VDD demonstrates across nine complex behavior learning tasks, that it is able to: i) accurately distill complex distributions learned by the diffusion model, ii) outperform existing state-of-the-art distillation methods, and iii) surpass conventional methods for training MoE. The code and videos are available at https://***/vdd-website. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Variational distillation of diffusion policies into mixture of experts 24

Variational distillation of diffusion policies into mixture ...

引用

Proceedings of the 38th International Conference on Neural Information Processing Systems

作者： Hongyi Zhou Denis Blessing Ge Li Onur Celik Xiaogang Jia Gerhard Neumann Rudolf Lioutikov Intuitive Robots Lab Karlsruhe Institute of Technology Autonomous Learning Robots Karlsruhe Institute of Technology Autonomous Learning Robots Karlsruhe Institute of Technology and FZI Research Center for Information Technology Intuitive Robots Lab Karlsruhe Institute of Technology and Autonomous Learning Robots Karlsruhe Institute of Technology

ISBN: (纸本)9798331314385

关键词：

来源：评论

学校读者我要写书评

暂无评论

ADVERSARIAL IMITATION learning WITH PREFERENCES 11

ADVERSARIAL IMITATION LEARNING WITH PREFERENCES

引用

11th International Conference on learning Representations, ICLR 2023

作者： Taranovic, Aleksandar Kupcsik, Andras Freymuth, Niklas Neumann, Gerhard Autonomous Learning Robots Lab Karlsruhe Institute of Technology Karlsruhe Germany Bosch Center for Artificial Intelligence Renningen Germany

Designing an accurate and explainable reward function for many Reinforcement learning tasks is a cumbersome and tedious process. Instead, learning policies directly from the feedback of human teachers naturally integrates human domain knowledge into the policy optimization process. Different feedback modalities, such as demonstrations and preferences, provide distinct benefits and disadvantages. For example, demonstrations convey a lot of information about the task but are often hard or costly to obtain from real experts while preferences typically contain less information but are in most cases cheap to generate. However, existing methods centered around human feedback mostly focus on a single teaching modality, causing them to miss out on important training data while making them less intuitive to use. In this paper we propose a novel method for policy learning that incorporates two different feedback types, namely demonstrations and preferences. To this end, we make use of the connection between discriminator training and density ratio estimation to incorporate preferences into the popular Adversarial Imitation learning paradigm. This insight allows us to express loss functions over both demonstrations and preferences in a unified framework. Besides expert demonstrations, we are also able to learn from imperfect ones and combine them with preferences to achieve improved task performance. We experimentally validate the effectiveness of combining both preferences and demonstrations on common benchmarks and also show that our method can efficiently learn challenging robot manipulation tasks. © 2023 11th International Conference on learning Representations, ICLR 2023. All rights reserved.

关键词： Demonstrations

来源：评论

学校读者我要写书评

暂无评论

Deep Black-Box Reinforcement learning with Movement Primitives 6

Deep Black-Box Reinforcement Learning with Movement Primitiv...

引用

6th Conference on Robot learning, CoRL 2022

作者： Otto, Fabian Celik, Onur Zhou, Hongyi Ziesche, Hanna Vien, Ngo Anh Neumann, Gerhard Bosch Center for Artificial Intelligence Germany University of Tübingen Germany Autonomous Learning Robots Karlsruhe Institute of Technology Germany

Episode-based reinforcement learning (ERL) algorithms treat reinforcement learning (RL) as a black-box optimization problem where we learn to select a parameter vector of a controller, often represented as a movement primitive, for a given task descriptor called a context. ERL offers several distinct benefits in comparison to step-based RL. It generates smooth control trajectories, can handle non-Markovian reward definitions, and the resulting exploration in parameter space is well suited for solving sparse reward settings. Yet, the high dimensionality of the movement primitive parameters has so far hampered the effective use of deep RL methods. In this paper, we present a new algorithm for deep ERL. It is based on differentiable trust region layers, a successful on-policy deep RL algorithm. These layers allow us to specify trust regions for the policy update that are solved exactly for each state using convex optimization, which enables policies learning with the high precision required for the ERL. We compare our ERL algorithm to state-of-the-art step-based algorithms in many complex simulated robotic control tasks. In doing so, we investigate different reward formulations - dense, sparse, and non-Markovian. While step-based algorithms perform well only on dense rewards, ERL performs favorably on sparse and non-Markovian rewards. Moreover, our results show that the sparse and the non-Markovian rewards are also often better suited to define the desired behavior, allowing us to obtain considerably higher quality policies compared to step-based RL. © 2023 Proceedings of Machine learning Research. All rights reserved.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Acquiring diverse skills using curriculum reinforcement learning with mixture of experts 24

Acquiring diverse skills using curriculum reinforcement lear...

引用

Proceedings of the 41st International Conference on Machine learning

作者： Onur Celik Aleksandar Taranovic Gerhard Neumann Autonomous Learning Robots Karlsruhe Institute of Technology Karlsruhe Germany and FZI Research Center for Information Technology Karlsruhe Germany Autonomous Learning Robots Karlsruhe Institute of Technology Karlsruhe Germany

Reinforcement learning (RL) is a powerful approach for acquiring a good-performing policy. However, learning diverse skills is challenging in RL due to the commonly used Gaussian policy parameterization. We propose Diverse Skill learning (Di-SkilL), an RL method for learning diverse skills using Mixture of Experts, where each expert formalizes a skill as a contextual motion primitive. Di-SkilL optimizes each expert and its associate context distribution to a maximum entropy objective that incentivizes learning diverse skills in similar contexts. The perexpert context distribution enables automatic curricula learning, allowing each expert to focus on its best-performing sub-region of the context space. To overcome hard discontinuities and multi-modalities without any prior knowledge of the environment's unknown context probability space, we leverage energy-based models to represent the per-expert context distributions and demonstrate how we can efficiently train them using the standard policy gradient objective. We show on challenging robot simulation tasks that Di-SkilL can learn diverse and performant skills.

关键词：

来源：评论

学校读者我要写书评

暂无评论

MaIL: Improving Imitation learning with Mamba

arXiv

引用

arXiv 2024年

This work presents Mamba Imitation learning (MaIL), a novel imitation learning (IL) architecture that provides an alternative to state-of-the-art (SoTA) Transformer-based policies. MaIL leverages Mamba, a state-space model designed to selectively focus on key features of the data. While Transformers are highly effective in data-rich environments due to their dense attention mechanisms, they can struggle with smaller datasets, often leading to overfitting or suboptimal representation learning. In contrast, Mamba’s architecture enhances representation learning efficiency by focusing on key features and reducing model complexity. This approach mitigates overfitting and enhances generalization, even when working with limited data. Extensive evaluations on the LIBERO benchmark demonstrate that MaIL consistently outperforms Transformers on all LIBERO tasks with limited data and matches their performance when the full dataset is available. Additionally, MaIL’s effectiveness is validated through its superior performance in three real robot experiments. Our code is available at https://***/ALRhub/MaIL. © 2024, CC BY.

关键词： Contrastive learning

来源：评论

学校读者我要写书评

暂无评论

Information maximizing curriculum: a curriculum-based approach for imitating diverse skills 23

Information maximizing curriculum: a curriculum-based approa...

引用

Proceedings of the 37th International Conference on Neural Information Processing Systems

作者： Denis Blessing Onur Celik Xiaogang Jia Moritz Reuss Maximilian Xiling Li Rudolf Lioutikov Gerhard Neumann Autonomous Learning Robots Karlsruhe Institute of Technology Autonomous Learning Robots Karlsruhe Institute of Technology and FZI Research Center for Information Technology Autonomous Learning Robots Karlsruhe Institute of Technology and Intuitive Robots Lab Karlsruhe Institute of Technology Intuitive Robots Lab Karlsruhe Institute of Technology

Imitation learning uses data for training policies to solve complex tasks. However, when the training data is collected from human demonstrators, it often leads to multimodal distributions because of the variability in human actions. Most imitation learning methods rely on a maximum likelihood (ML) objective to learn a parameterized policy, but this can result in suboptimal or unsafe behavior due to the mode-averaging property of the ML objective. In this work, we propose Information Maximizing Curriculum, a curriculum-based approach that assigns a weight to each data point and encourages the model to specialize in the data it can represent, effectively mitigating the mode-averaging problem by allowing the model to ignore data from modes it cannot represent. To cover all modes and thus, enable diverse behavior, we extend our approach to a mixture of experts (MoE) policy, where each mixture component selects its own subset of the training data for learning. A novel, maximum entropy-based objective is proposed to achieve full coverage of the dataset, thereby enabling the policy to encompass all modes within the data distribution. We demonstrate the effectiveness of our approach on complex simulated control tasks using diverse human demonstrations, achieving superior performance compared to state-of-the-art methods.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Beyond ELBOs: a large-scale evaluation of variational methods for sampling 24

Beyond ELBOs: a large-scale evaluation of variational method...

引用

Proceedings of the 41st International Conference on Machine learning

作者： Denis Blessing Xiaogang Jia Johannes Esslinger Francisco Vargas Gerhard Neumann Autonomous Learning Robots Karlsruhe Institute of Technology Karlsruhe Germany University of Cambridge Cambridge UK Autonomous Learning Robots Karlsruhe Institute of Technology Karlsruhe Germany and FZI Research Center for Information Technology Karlsruhe Germany

Monte Carlo methods, Variational Inference, and their combinations play a pivotal role in sampling from intractable probability distributions. However, current studies lack a unified evaluation framework, relying on disparate performance measures and limited method comparisons across diverse tasks, complicating the assessment of progress and hindering the decision-making of practitioners. In response to these challenges, our work introduces a benchmark that evaluates sampling methods using a standardized task suite and a broad range of performance criteria. Moreover, we study existing metrics for quantifying mode collapse and introduce novel metrics for this purpose. Our findings provide insights into strengths and weaknesses of existing sampling methods, serving as a valuable reference for future developments. The code is publicly available here.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：