检索结果-内蒙古大学图书馆

Diffusion for Multi-Embodiment Grasping

IEEE ROBOTICS AND AUTOMATION LETTERS 2025年第3期10卷 2694-2701页

作者： Freiberg, Roman Qualmann, Alexander Vien, Ngo Anh Neumann, Gerhard BOSCH Corp Res D-71272 Renningen Germany Karlsruhe Inst Technol Autonomous Learning Robots Lab D-76137 Karlsruhe Germany

Grasping is a fundamental skill in robotics with diverse applications. However, current approaches for predicting successful grasps are often tailored to specific grippers, limiting their applicability when gripper designs change. To address this limitation, we explore the transfer of grasping strategies between various gripper designs, enabling the use of data from diverse sources. We present an approach based on equivariant diffusion that facilitates gripper-agnostic encoding of scenes containing graspable objects and gripper-aware decoding of grasp poses by integrating gripper geometry into the model. We also develop a dataset generation framework that produces cluttered scenes with variable-sized object heaps, improving the training of grasp synthesis methods. Experimental evaluation on diverse object datasets demonstrates the generalizability of our approach across gripper architectures, ranging from simple parallel-jaw grippers to humanoid hands, outperforming both single-gripper and multi-gripper State-of-the-Art methods.

关键词： Grippers Grasping robots Point cloud compression Manifolds Hands Training Robot vision systems Pipelines Internet deep learning in grasping and manipulation transfer learning

来源：评论

学校读者我要写书评

暂无评论

MuTT: A Multimodal Trajectory Transformer for Robot Skills

MuTT: A Multimodal Trajectory Transformer for Robot Skills

引用

2024 International Conference on Intelligent robots and Systems

作者： Kienle, Claudius Alt, Benjamin Celik, Onur Becker, Philipp Katic, Darko Jaekel, Rainer Neumann, Gerhard ArtiMinds Robot Karlsruhe Germany KIT Autonomous & Learning Robots Karlsruhe Germany

ISBN: (纸本)9798350377712;9798350377705

High-level robot skills represent an increasingly popular paradigm in robot programming. However, configuring the skills' parameters for a specific task remains a manual and time-consuming endeavor. Existing approaches for learning or optimizing these parameters often require numerous real-world executions or do not work in dynamic environments. To address these challenges, we propose Multimodal Trajectory Transformer (MuTT), a novel encoder-decoder transformer architecture designed to predict environment-aware executions of robot skills by integrating vision, trajectory, and robot skill parameters. Notably, we pioneer the fusion of vision and trajectory, introducing a novel trajectory projection. Furthermore, we illustrate MuTT's efficacy as a predictor when combined with a model-based robot skill optimizer. This approach facilitates the optimization of robot skill parameters for the current environment, without the need for real-world executions during optimization. Designed for compatibility with any representation of robot skills, MuTT demonstrates its versatility across three comprehensive experiments, showcasing superior performance across two different skill representations.

关键词： Robot programming

来源：评论

学校读者我要写书评

暂无评论

Physics-informed MESHGRAPHNETS (PI-MGNs): Neural finite element solvers for non-stationary and nonlinear simulations on arbitrary meshes

引用

COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING 2024年 429卷

作者： Wuerth, Tobias Freymuth, Niklas Zimmerling, Clemens Neumann, Gerhard Kaerger, Luise Karlsruhe Inst Technol KIT Inst Vehicle Syst Technol Karlsruhe Germany Karlsruhe Inst Technol KIT Autonomous Learning Robots Karlsruhe Germany

Engineering components must meet increasing technological demands in ever shorter development cycles. To face these challenges, a holistic approach is essential that allows for the concurrent development of part design, material system and manufacturing process. Current approaches employ numerical simulations, which quickly becomes computation -intensive, especially for iterative optimization. Data -driven machine learning methods can be used to replace time- and resource -intensive numerical simulations. In particular, MESHGRAPHNETS (MGNs) have shown promising results. They enable fast and accurate predictions on unseen mesh geometries while being fully differentiable for optimization. However, these models rely on large amounts of expensive training data, such as numerical simulations. Physics -informed neural networks (PINNs) offer an opportunity to train neural networks with partial differential equations instead of labeled data, but have not yet been extended to handle time -dependent simulations of arbitrary meshes. This work introduces PI-MGNs, a hybrid approach that combines PINNs and MGNs to quickly and accurately solve non -stationary and nonlinear partial differential equations (PDEs) on arbitrary meshes. The method is exemplified for thermal process simulations of unseen parts with inhomogeneous material distribution. Further results show that the model scales well to large and complex meshes after being trained exclusively on small, generic meshes.

关键词： Graph neural network Machine learning Physics-based simulation Surrogate model Partial differential equations

来源：评论

学校读者我要写书评

暂无评论

MaIL: Improving Imitation learning with Selective State Space Models 8

MaIL: Improving Imitation Learning with Selective State Spac...

引用

8th Conference on Robot learning, CoRL 2024

作者： Jia, Xiaogang Wang, Qian Donat, Atalay Xing, Bowen Li, Ge Zhou, Hongyi Celik, Onur Blessing, Denis Lioutikov, Rudolf Neumann, Gerhard Autonomous Learning Robots Karlsruhe Institute of Technology Germany Intuitive Robots Lab Karlsruhe Institute of Technology Germany

This work presents Mamba Imitation learning (MaIL), a novel imitation learning (IL) architecture that provides an alternative to state-of-the-art (SoTA) Transformer-based policies. MaIL leverages Mamba, a state-space model designed to selectively focus on key features of the data. While Transformers are highly effective in data-rich environments due to their dense attention mechanisms, they can struggle with smaller datasets, often leading to overfitting or suboptimal representation learning. In contrast, Mamba's architecture enhances representation learning efficiency by focusing on key features and reducing model complexity. This approach mitigates overfitting and enhances generalization, even when working with limited data. Extensive evaluations on the LIBERO benchmark demonstrate that MaIL consistently outperforms Transformers on all LIBERO tasks with limited data and matches their performance when the full dataset is available. Additionally, MaIL's effectiveness is validated through its superior performance in three real robot experiments. Our code is available at https://***/ALRhub/MaIL. © 2024 Proceedings of Machine learning Research.

关键词： Contrastive learning

来源：评论

学校读者我要写书评

暂无评论

TOWARDS DIVERSE BEHAVIORS: A BENCHMARK FOR IMITATION learning WITH HUMAN DEMONSTRATIONS 12

TOWARDS DIVERSE BEHAVIORS: A BENCHMARK FOR IMITATION LEARNIN...

引用

12th International Conference on learning Representations, ICLR 2024

作者： Jia, Xiaogang Blessing, Denis Jiang, Xinkai Reuss, Moritz Donat, Atalay Lioutikov, Rudolf Neumann, Gerhard Autonomous Learning Robots Karlsruhe Institute of Technology Germany Intuitive Robots Lab Karlsruhe Institute of Technology Germany

Imitation learning with human data has demonstrated remarkable success in teaching robots in a wide range of skills. However, the inherent diversity in human behavior leads to the emergence of multi-modal data distributions, thereby presenting a formidable challenge for existing imitation learning algorithms. Quantifying a model's capacity to capture and replicate this diversity effectively is still an open problem. In this work, we introduce simulation benchmark environments and the corresponding Datasets with Diverse human Demonstrations for Imitation learning (D3IL), designed explicitly to evaluate a model's ability to learn multi-modal behavior. Our environments are designed to involve multiple sub-tasks that need to be solved, consider manipulation of multiple objects which increases the diversity of the behavior and can only be solved by policies that rely on closed loop sensory feedback. Other available datasets are missing at least one of these challenging properties. To address the challenge of diversity quantification, we introduce tractable metrics that provide valuable insights into a model's ability to acquire and reproduce diverse behaviors. These metrics offer a practical means to assess the robustness and versatility of imitation learning algorithms. Furthermore, we conduct a thorough evaluation of state-of-the-art methods on the proposed task suite. This evaluation serves as a benchmark for assessing their capability to learn diverse behaviors. Our findings shed light on the effectiveness of these methods in tackling the intricate problem of capturing and generalizing multi-modal human behaviors, offering a valuable reference for the design of future imitation learning algorithms. Project page: https://***/d3il-website/. © 2024 12th International Conference on learning Representations, ICLR 2024. All rights reserved.

关键词： learning algorithms

来源：评论

学校读者我要写书评

暂无评论

Curriculum-Based Imitation of Versatile Skills

Curriculum-Based Imitation of Versatile Skills

引用

IEEE International Conference on Robotics and Automation (ICRA)

作者： Li, Maximilian Xiling Celik, Onur Becker, Philipp Blessing, Denis Lioutikov, Rudolf Neumann, Gerhard Karlsruhe Inst Technol Intuit Robots Lab Karlsruhe Germany Karlsruhe Inst Technol Autonomous Learning Robots Karlsruhe Germany

ISBN: (纸本)9798350323658

learning skills by imitation is a promising concept for the intuitive teaching of robots. A common way to learn such skills is to learn a parametric model by maximizing the likelihood given the demonstrations. Yet, human demonstrations are often multi-modal, i.e., the same task is solved in multiple ways which is a major challenge for most imitation learning methods that are based on such a maximum likelihood (ML) objective. The ML objective forces the model to cover all data, it prevents specialization in the context space and can cause mode-averaging in the behavior space, leading to suboptimal or potentially catastrophic behavior. Here, we alleviate those issues by introducing a curriculum using a weight for each data point, allowing the model to specialize on data it can represent while incentivizing it to cover as much data as possible by an entropy bonus. We extend our algorithm to a Mixture of (linear) Experts (MoE) such that the single components can specialize on local context regions, while the MoE covers all data points. We evaluate our approach in complex simulated and real robot control tasks and show it learns from versatile human demonstrations and significantly outperforms current SOTA methods. (1)

关键词： Imitation Versatility Curriculum learning

来源：评论

学校读者我要写书评

暂无评论

Variational Distillation of Diffusion Policies into Mixture of Experts 38

Variational Distillation of Diffusion Policies into Mixture ...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Zhou, Hongyi Blessing, Denis Li, Ge Celik, Onur Jia, Xiaogang Neumann, Gerhard Lioutikov, Rudolf Intuitive Robots Lab Karlsruhe Institute of Technology Germany Autonomous Learning Robots Karlsruhe Institute of Technology Germany FZI Research Center for Information Technology Germany

This work introduces Variational Diffusion Distillation (VDD), a novel method that distills denoising diffusion policies into Mixtures of Experts (MoE) through variational inference. Diffusion Models are the current state-of-the-art in generative modeling due to their exceptional ability to accurately learn and represent complex, multi-modal distributions. This ability allows Diffusion Models to replicate the inherent diversity in human behavior, making them the preferred models in behavior learning such as learning from Human Demonstrations (LfD). However, diffusion models come with some drawbacks, including the intractability of likelihoods and long inference times due to their iterative sampling process. The inference times, in particular, pose a significant challenge to real-time applications such as robot control. In contrast, MoEs effectively address the aforementioned issues while retaining the ability to represent complex distributions but are notoriously difficult to train. VDD is the first method that distills pre-trained diffusion models into MoE models, and hence, combines the expressiveness of Diffusion Models with the benefits of Mixture Models. Specifically, VDD leverages a decompositional upper bound of the variational objective that allows the training of each expert separately, resulting in a robust optimization scheme for MoEs. VDD demonstrates across nine complex behavior learning tasks, that it is able to: i) accurately distill complex distributions learned by the diffusion model, ii) outperform existing state-of-the-art distillation methods, and iii) surpass conventional methods for training MoE. The code and videos are available at https://***/vdd-website. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Acquiring Diverse Skills using Curriculum Reinforcement learning with Mixture of Experts 41

Acquiring Diverse Skills using Curriculum Reinforcement Lear...

引用

41st International Conference on Machine learning (ICML)

作者： Celik, Onur Taranovic, Aleksandar Neu, Gerhard Karlsruhe Inst Technol Autonomous Learning Robots Karlsruhe Germany FZI Res Ctr Informat Technol Karlsruhe Germany

Reinforcement learning (RL) is a powerful approach for acquiring a good-performing policy. However, learning diverse skills is challenging in RL due to the commonly used Gaussian policy parameterization. We propose Diverse Skill learning (Di-SkilL1), an RL method for learning diverse skills using Mixture of Experts, where each expert formalizes a skill as a contextual motion primitive. Di-SkilL optimizes each expert and its associate context distribution to a maximum entropy objective that incentivizes learning diverse skills in similar contexts. The per-expert context distribution enables automatic curricula learning, allowing each expert to focus on its best-performing sub-region of the context space. To overcome hard discontinuities and multi-modalities without any prior knowledge of the environment's unknown context probability space, we leverage energy-based models to represent the per-expert context distributions and demonstrate how we can efficiently train them using the standard policy gradient objective. We show on challenging robot simulation tasks that Di-SkilL can learn diverse and performant skills.

关键词： Adversarial machine learning

来源：评论

学校读者我要写书评

暂无评论

SA6D: Self-Adaptive Few-Shot 6D Pose Estimator for Novel and Occluded Objects 7

SA6D: Self-Adaptive Few-Shot 6D Pose Estimator for Novel and...

引用

7th Conference on Robot learning (CoRL)

作者： Gao, Ning Vien, Ngo Anh Ziesche, Hanna Neumann, Gerhard Bosch Ctr Artificial Intelligence Tubingen Germany KIT Autonomous Learning Robots Karlsruhe Germany

6D pose estimation is one of the critical aspects in robotic manipulation. Most existing approaches have difficulties in extending predictions to scenarios where novel object instances are introduced, especially with heavy occlusions. In this work, we propose a few-shot pose estimation (FSPE) approach called SA6D on novel objects, which uses a self-adaptive segmentation module to identify the novel target object and construct a point cloud model of the target object using only a small number of cluttered reference images. Unlike existing methods, SA6D does not require object-centric reference images or any additional object information, making it a more generalizable and scalable solution across categories. We evaluate SA6D on real-world tabletop object datasets and demonstrate that SA6D outperforms existing FSPE methods, particularly in cluttered scenes with occlusions, while requiring fewer reference images.

关键词： Contrastive learning 6D Pose Estimation Self-Adaptation

来源：评论

学校读者我要写书评

暂无评论

Domain-Specific Fine-Tuning of Large Language Models for Interactive Robot Programming 15th

Domain-Specific Fine-Tuning of Large Language Models for Int...

引用

15th European Robotics Forum

作者： Alt, Benjamin Kellner, Urs Taranovic, Aleksandar Katic, Darko Hermann, Andreas Jaekle, Rainer Neumann, Gerhard ArtiMinds Robot D-76131 Karlsruhe Germany Karlsruhe Inst Technol Autonomous Learning Robots Lab D-76131 Karlsruhe Germany

ISBN: (纸本)9783031764264;9783031764240;9783031764233

Industrial robots are applied in a widening range of industries, but robot programming mostly remains a task limited to programming experts. We propose a natural language-based assistant for programming of advanced, industrial robotic applications and investigate strategies for domain-specific fine-tuning of foundation models with limited data and compute.

关键词： large language models robot programming fine-tuning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：