检索结果-内蒙古大学图书馆

41st International Conference on Machine learning (ICML)

作者： Blessing, Denis Jia, Xiaogang Esslinger, Johannes Vargas, Francisco Neumann, Gerhard Karlsruhe Inst Technol Autonomous Learning Robots Karlsruhe Germany Univ Cambridge Cambridge England FZI Res Ctr Informat Technol Karlsruhe Germany

Monte Carlo methods, Variational Inference, and their combinations play a pivotal role in sampling from intractable probability distributions. However, current studies lack a unified evaluation framework, relying on disparate performance measures and limited method comparisons across diverse tasks, complicating the assessment of progress and hindering the decision-making of practitioners. In response to these challenges, our work introduces a benchmark that evaluates sampling methods using a standardized task suite and a broad range of performance criteria. Moreover, we study existing metrics for quantifying mode collapse and introduce novel metrics for this purpose. Our findings provide insights into strengths and weaknesses of existing sampling methods, serving as a valuable reference for future developments. The code is publicly available here.

关键词： Monte Carlo methods

来源：评论

学校读者我要写书评

暂无评论

Information Maximizing Curriculum: A Curriculum-Based Approach for Imitating Diverse Skills 37

Information Maximizing Curriculum: A Curriculum-Based Approa...

引用

37th Conference on Neural Information Processing Systems (NeurIPS)

作者： Blessing, Denis Celik, Onur Jia, Xiaogang Reuss, Moritz Li, Maximilian Xiling Lioutikov, Rudolf Neumann, Gerhard Karlsruhe Inst Technol Autonomous Learning Robots Karlsruhe Germany Karlsruhe Inst Technol Intuit Robots Lab Karlsruhe Germany FZI Res Ctr Informat Technol Karlsruhe Germany

ISBN: (纸本)9781713899921

Imitation learning uses data for training policies to solve complex tasks. However, when the training data is collected from human demonstrators, it often leads to multimodal distributions because of the variability in human actions. Most imitation learning methods rely on a maximum likelihood (ML) objective to learn a parameterized policy, but this can result in suboptimal or unsafe behavior due to the mode-averaging property of the ML objective. In this work, we propose Information Maximizing Curriculum, a curriculum-based approach that assigns a weight to each data point and encourages the model to specialize in the data it can represent, effectively mitigating the mode-averaging problem by allowing the model to ignore data from modes it cannot represent. To cover all modes and thus, enable diverse behavior, we extend our approach to a mixture of experts (MoE) policy, where each mixture component selects its own subset of the training data for learning. A novel, maximum entropy-based objective is proposed to achieve full coverage of the dataset, thereby enabling the policy to encompass all modes within the data distribution. We demonstrate the effectiveness of our approach on complex simulated control tasks using diverse human demonstrations, achieving superior performance compared to state-of-the-art methods.

关键词： Demonstrations

来源：评论

学校读者我要写书评

暂无评论

Variational distillation of diffusion policies into mixture of experts 24

Variational distillation of diffusion policies into mixture ...

引用

Proceedings of the 38th International Conference on Neural Information Processing Systems

作者： Hongyi Zhou Denis Blessing Ge Li Onur Celik Xiaogang Jia Gerhard Neumann Rudolf Lioutikov Intuitive Robots Lab Karlsruhe Institute of Technology Autonomous Learning Robots Karlsruhe Institute of Technology Autonomous Learning Robots Karlsruhe Institute of Technology and FZI Research Center for Information Technology Intuitive Robots Lab Karlsruhe Institute of Technology and Autonomous Learning Robots Karlsruhe Institute of Technology

ISBN: (纸本)9798331314385

This work introduces Variational Diffusion Distillation (VDD), a novel method that distills denoising diffusion policies into Mixtures of Experts (MoE) through variational inference. Diffusion Models are the current state-of-the-art in generative modeling due to their exceptional ability to accurately learn and represent complex, multi-modal distributions. This ability allows Diffusion Models to replicate the inherent diversity in human behavior, making them the preferred models in behavior learning such as learning from Human Demonstrations (LfD). However, diffusion models come with some drawbacks, including the intractability of likelihoods and long inference times due to their iterative sampling process. The inference times, in particular, pose a significant challenge to real-time applications such as robot control. In contrast, MoEs effectively address the aforementioned issues while retaining the ability to represent complex distributions but are notoriously difficult to train. VDD is the first method that distills pre-trained diffusion models into MoE models, and hence, combines the expressiveness of Diffusion Models with the benefits of Mixture Models. Specifically, VDD leverages a decompositional upper bound of the variational objective that allows the training of each expert separately, resulting in a robust optimization scheme for MoEs. VDD demonstrates across nine complex behavior learning tasks, that it is able to: i) accurately distill complex distributions learned by the diffusion model, ii) outperform existing state-of-the-art distillation methods, and iii) surpass conventional methods for training MoE. The code and videos are available at https://***/vdd-website.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Swarm Reinforcement learning for Adaptive Mesh Refinement 37

Swarm Reinforcement Learning for Adaptive Mesh Refinement

引用

37th Conference on Neural Information Processing Systems (NeurIPS)

作者： Freymuth, Niklas Dahlinger, Philipp Wuerth, Tobias Reisch, Simon Kaerger, Luise Neumann, Gerhard Karlsruhe Inst Technol Autonomous Learning Robots Karlsruhe Germany Karlsruhe Inst Technol Inst Vehicle Syst Technol Karlsruhe Germany

ISBN: (纸本)9781713899921

Adaptive Mesh Refinement (AMR) enhances the Finite Element Method, an important technique for simulating complex problems in engineering, by dynamically refining mesh regions, enabling a favorable trade-off between computational speed and simulation accuracy. Classical methods for AMR depend on heuristics or expensive error estimators, hindering their use for complex simulations. Recent learning-based AMR methods tackle these issues, but so far scale only to simple toy examples. We formulate AMR as a novel Adaptive Swarm Markov Decision Process in which a mesh is modeled as a system of simple collaborating agents that may split into multiple new agents. This framework allows for a spatial reward formulation that simplifies the credit assignment problem, which we combine with Message Passing Networks to propagate information between neighboring mesh elements. We experimentally validate our approach, Adaptive Swarm Mesh Refinement (ASMR), on challenging refinement tasks. Our approach learns reliable and efficient refinement strategies that can robustly generalize to different domains during inference. Additionally, it achieves a speedup of up to 2 orders of magnitude compared to uniform refinements in more demanding simulations. We outperform learned baselines and heuristics, achieving a refinement quality that is on par with costly error-based oracle AMR strategies.

关键词： Markov processes

来源：评论

学校读者我要写书评

暂无评论

Deep Black-Box Reinforcement learning with Movement Primitives 6

Deep Black-Box Reinforcement Learning with Movement Primitiv...

引用

6th Conference on Robot learning, CoRL 2022

作者： Otto, Fabian Celik, Onur Zhou, Hongyi Ziesche, Hanna Vien, Ngo Anh Neumann, Gerhard Bosch Center for Artificial Intelligence Germany University of Tübingen Germany Autonomous Learning Robots Karlsruhe Institute of Technology Germany

Episode-based reinforcement learning (ERL) algorithms treat reinforcement learning (RL) as a black-box optimization problem where we learn to select a parameter vector of a controller, often represented as a movement primitive, for a given task descriptor called a context. ERL offers several distinct benefits in comparison to step-based RL. It generates smooth control trajectories, can handle non-Markovian reward definitions, and the resulting exploration in parameter space is well suited for solving sparse reward settings. Yet, the high dimensionality of the movement primitive parameters has so far hampered the effective use of deep RL methods. In this paper, we present a new algorithm for deep ERL. It is based on differentiable trust region layers, a successful on-policy deep RL algorithm. These layers allow us to specify trust regions for the policy update that are solved exactly for each state using convex optimization, which enables policies learning with the high precision required for the ERL. We compare our ERL algorithm to state-of-the-art step-based algorithms in many complex simulated robotic control tasks. In doing so, we investigate different reward formulations - dense, sparse, and non-Markovian. While step-based algorithms perform well only on dense rewards, ERL performs favorably on sparse and non-Markovian rewards. Moreover, our results show that the sparse and the non-Markovian rewards are also often better suited to define the desired behavior, allowing us to obtain considerably higher quality policies compared to step-based RL. © 2023 Proceedings of Machine learning Research. All rights reserved.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

ADVERSARIAL IMITATION learning WITH PREFERENCES 11

ADVERSARIAL IMITATION LEARNING WITH PREFERENCES

引用

11th International Conference on learning Representations, ICLR 2023

作者： Taranovic, Aleksandar Kupcsik, Andras Freymuth, Niklas Neumann, Gerhard Autonomous Learning Robots Lab Karlsruhe Institute of Technology Karlsruhe Germany Bosch Center for Artificial Intelligence Renningen Germany

Designing an accurate and explainable reward function for many Reinforcement learning tasks is a cumbersome and tedious process. Instead, learning policies directly from the feedback of human teachers naturally integrates human domain knowledge into the policy optimization process. Different feedback modalities, such as demonstrations and preferences, provide distinct benefits and disadvantages. For example, demonstrations convey a lot of information about the task but are often hard or costly to obtain from real experts while preferences typically contain less information but are in most cases cheap to generate. However, existing methods centered around human feedback mostly focus on a single teaching modality, causing them to miss out on important training data while making them less intuitive to use. In this paper we propose a novel method for policy learning that incorporates two different feedback types, namely demonstrations and preferences. To this end, we make use of the connection between discriminator training and density ratio estimation to incorporate preferences into the popular Adversarial Imitation learning paradigm. This insight allows us to express loss functions over both demonstrations and preferences in a unified framework. Besides expert demonstrations, we are also able to learn from imperfect ones and combine them with preferences to achieve improved task performance. We experimentally validate the effectiveness of combining both preferences and demonstrations on common benchmarks and also show that our method can efficiently learn challenging robot manipulation tasks. © 2023 11th International Conference on learning Representations, ICLR 2023. All rights reserved.

关键词： Demonstrations

来源：评论

学校读者我要写书评

暂无评论

What Matters For Meta-learning Vision Regression Tasks?

What Matters For Meta-Learning Vision Regression Tasks?

引用

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Gao, Ning Ziesche, Hanna Ngo Anh Vien Volpp, Michael Neumann, Gerhard Bosch Ctr Artificial Intelligence Stuttgart Germany KIT Autonomous Learning Robots Karlsruhe Germany

ISBN: (数字)9781665469463

ISBN: (纸本)9781665469463

Meta-learning is widely used in few-shot classification and function regression due to its ability to quickly adapt to unseen tasks. However, it has not yet been well explored on regression tasks with high dimensional inputs such as images. This paper makes two main contributions that help understand this barely explored area. First, we design two new types of cross-category level vision regression tasks, namely object discovery and pose estimation of unprecedented complexity in the meta-learning domain for computer vision. To this end, we (i) exhaustively evaluate common meta-learning techniques on these tasks, and (ii) quantitatively analyze the effect of various deep learning techniques commonly used in recent meta-learning algorithms in order to strengthen the generalization capability: data augmentation, domain randomization, task augmentation and meta-regularization. Finally, we (iii) provide some insights and practical recommendations for training meta-learning algorithms on vision regression tasks. Second, we propose the addition of functional contrastive learning (FCL) over the task representations in Conditional Neural Processes (CNPs) and train in an end-to-end fashion. The experimental results show that the results of prior work are misleading as a consequence of a poor choice of the loss function as well as too small meta-training sets. Specifically, we find that CNPs outperform MAML on most tasks without fine-tuning. Furthermore, we observe that naive task augmentation without a tailored design results in underfitting.

关键词： Training Deep learning Computer vision Pose estimation Pattern recognition Complexity theory Task analysis

来源：评论

学校读者我要写书评

暂无评论

Specializing Versatile Skill Libraries using Local Mixture of Experts 5

Specializing Versatile Skill Libraries using Local Mixture o...

引用

5th Conference on Robot learning (CoRL)

作者： Celik, Onur Zhou, Dongzhuoran Li, Ge Becker, Philipp Neumann, Gerhard KIT Autonomous Learning Robots Karlsruhe Germany

A long-cherished vision in robotics is to equip robots with skills that match the versatility and precision of humans. For example, when playing table tennis, a robot should be capable of returning the ball in various ways while precisely placing it at the desired location. A common approach to model such versatile behavior is to use a Mixture of Experts (MoE) model, where each expert is a contextual motion primitive. However, learning such MoEs is challenging as most objectives force the model to cover the entire context space, which prevents specialization of the primitives resulting in rather low-quality components. Starting from maximum entropy reinforcement learning (RL), we decompose the objective into optimizing an individual lower bound per mixture component. Further, we introduce a curriculum by allowing the components to focus on a local context region, enabling the model to learn highly accurate skill representations. To this end, we use local context distributions that are adapted jointly with the expert primitives. Our lower bound advocates an iterative addition of new components, where new components will concentrate on local context regions not covered by the current MoE. This local and incremental learning results in a modular MoE model of high accuracy and versatility, where both properties can be scaled by adding more components on the fly. We demonstrate this by an extensive ablation and on two challenging simulated robot skill learning tasks. We compare our achieved performance to LaDiPS and HiREPS, a known hierarchical policy search method for learning diverse skills.

关键词： Curriculum Versatile Skill learning Episodic Hierarchical RL

来源：评论

学校读者我要写书评

暂无评论

Inferring Versatile Behavior from Demonstrations by Matching Geometric Descriptors 6

Inferring Versatile Behavior from Demonstrations by Matching...

引用

6th Conference on Robot learning (CoRL)

作者： Freymuth, Niklas Schreiber, Nicolas Becker, Philipp Taranovic, Aleksandar Neumann, Gerhard Karlsruhe Inst Technol Autonomous Learning Robots Karlsruhe Germany Bosch Ctr Artificial Intelligence Renningen Germany

Humans intuitively solve tasks in versatile ways, varying their behavior in terms of trajectory-based planning and for individual steps. Thus, they can easily generalize and adapt to new and changing environments. Current Imitation learning algorithms often only consider unimodal expert demonstrations and act in a state-action-based setting, making it difficult for them to imitate human behavior in case of versatile demonstrations. Instead, we combine a mixture of movement primitives with a distribution matching objective to learn versatile behaviors that match the expert's behavior and versatility. To facilitate generalization to novel task configurations, we do not directly match the agent's and expert's trajectory distributions but rather work with concise geometric descriptors which generalize well to unseen task configurations. We empirically validate our method on various robot tasks using versatile human demonstrations and compare to imitation learning algorithms in a state-action setting as well as a trajectory-based setting. We find that the geometric descriptors greatly help in generalizing to new task configurations and that combining them with our distribution-matching objective is crucial for representing and reproducing versatile behavior.

关键词： Imitation learning Versatile Skill learning Distribution Matching

来源：评论

学校读者我要写书评

暂无评论

MuTT: A Multimodal Trajectory Transformer for Robot Skills

arXiv

引用

arXiv 2024年

作者： Kienle, Claudius Alt, Benjamin Celik, Onur Becker, Philipp Katic, Darko Jäkel, Rainer Neumann, Gerhard ArtiMinds Robotics Karlsruhe Germany Autonomous Learning Robots KIT Germany

High-level robot skills represent an increasingly popular paradigm in robot programming. However, configuring the skills’ parameters for a specific task remains a manual and time-consuming endeavor. Existing approaches for learning or optimizing these parameters often require numerous real-world executions or do not work in dynamic environments. To address these challenges, we propose Multimodal Trajectory Transformer (MuTT), a novel encoder-decoder transformer architecture designed to predict environment-aware executions of robot skills by integrating vision, trajectory, and robot skill parameters. Notably, we pioneer the fusion of vision and trajectory, introducing a novel trajectory projection. Furthermore, we illustrate MuTT’s efficacy as a predictor when combined with a model-based robot skill optimizer. This approach facilitates the optimization of robot skill parameters for the current environment, without the need for real-world executions during optimization. Designed for compatibility with any representation of robot skills, MuTT demonstrates its versatility across three comprehensive experiments, showcasing superior performance across two different skill representations. © 2024, CC BY-NC-SA.

关键词： Robot programming

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：