检索结果-内蒙古大学图书馆

Adaptive Swarm Mesh Refinement using Deep Reinforcement learning with Local Rewards

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Freymuth, Niklas Dahlinger, Philipp Würth, Tobias Reisch, Simon Kärger, Luise Neumann, Gerhard Autonomous Learning Robots Karlsruhe Institute of Technology Karlsruhe Germany Institute of Vehicle Systems Technology Karlsruhe Institute of Technology Karlsruhe Germany

Simulating physical systems is essential in engineering, but analytical solutions are limited to straightforward problems. Consequently, numerical methods like the Finite Element Method (FEM) are widely used. However, the FEM becomes computationally expensive as problem complexity and accuracy demands increase. Adaptive Mesh Refinement (AMR) improves the FEM by dynamically allocating mesh elements on the domain, balancing computational speed and accuracy. Classical AMR depends on heuristics or expensive error estimators, limiting its use in complex simulations. While learning-based AMR methods are promising, they currently only scale to simple problems. In this work, we formulate AMR as a system of collaborating, homogeneous agents that iteratively split into multiple new agents. This agent-wise perspective enables a spatial reward formulation focused on reducing the maximum mesh element error. Our approach, Adaptive Swarm Mesh Refinement (ASMR), offers efficient, stable optimization and generates highly adaptive meshes at user-defined resolution during inference. Extensive experiments, including volumetric meshes and Neumann boundary conditions, demonstrate that ASMR exceeds heuristic approaches and learned baselines, matching the performance of expensive error-based oracle AMR strategies. ASMR additionally generalizes to different domains during inference, and produces meshes that simulate up to 2 orders of magnitude faster than uniform refinements in more demanding settings. © 2024, CC BY.

关键词： Errors

End-to-End learning of Hybrid Inverse Dynamics Models for Precise and Compliant Impedance Control 18th

学校读者我要写书评

暂无评论

End-to-End Learning of Hybrid Inverse Dynamics Models for Pr...

18th Robotics: Science and Systems, RSS 2022

作者： Reuss, Moritz van Duijkeren, Niels Krug, Robert Becker, Philipp Shaj, Vaisakh Neumann, Gerhard Bosch Corporate Research Renningen Germany LCAS University Of Lincoln United Kingdom Intuitive Robots Lab Karlsruhe Institute of Technology Germany Autonomous Learning Robots Karlsruhe Institute of Technology Germany

ISBN: (纸本)9780992374785

It is well-known that inverse dynamics models can improve tracking performance in robot control. These models need to precisely capture the robot dynamics, which consist of well-understood components, e.g., rigid body dynamics, and effects that remain challenging to capture, e.g., stick-slip friction and mechanical flexibilities. Such effects exhibit hysteresis and partial observability, rendering them, particularly challenging to model. Hence, hybrid models, which combine a physical prior with data-driven approaches are especially well-suited in this setting. We present a novel hybrid model formulation that enables us to identify fully physically consistent inertial parameters of a rigid body dynamics model which is paired with a recurrent neural network architecture, allowing us to capture unmodeled partially observable effects using the network memory. We compare our approach against state-of-the-art inverse dynamics models on a 7 degree of freedom manipulator. Using data sets obtained through an optimal experiment design approach, we study the accuracy of offline torque prediction and generalization capabilities of joint learning methods. In control experiments on the real system, we evaluate the model as a feed-forward term for impedance control and show the feedback gains can be drastically reduced to achieve a given tracking accuracy. © 2022, MIT Press Journals. All rights reserved.

关键词： learning systems

END-TO-END learning OF GAUSSIAN MIXTURE PRIORS FOR DIFFUSION SAMPLER

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Blessing, Denis Jia, Xiaogang Neumann, Gerhard Autonomous Learning Robots Karlsruhe Institute of Technology Germany FZI Research Center for Information Technology Germany

Diffusion models optimized via variational inference (VI) have emerged as a promising tool for generating samples from unnormalized target densities. These models create samples by simulating a stochastic differential equation, starting from a simple, tractable prior, typically a Gaussian distribution. However, when the support of this prior differs greatly from that of the target distribution, diffusion models often struggle to explore effectively or suffer from large discretization errors. Moreover, learning the prior distribution can lead to mode-collapse, exacerbated by the mode-seeking nature of reverse Kullback-Leibler divergence commonly used in VI. To address these challenges, we propose end-to-end learnable Gaussian mixture priors (GMPs). GMPs offer improved control over exploration, adaptability to target support, and increased expressiveness to counteract mode collapse. We further leverage the structure of mixture models by proposing a strategy to iteratively refine the model by adding mixture components during training. Our experimental results demonstrate significant performance improvements across a diverse range of real-world and synthetic benchmark problems when using GMPs without requiring additional target evaluations. © 2025, CC BY.

关键词： Stochastic systems

Acquiring Diverse Skills using Curriculum Reinforcement learning with Mixture of Experts

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Celik, Onur Taranovic, Aleksandar Neumann, Gerhard Autonomous Learning Robots Karlsruhe Institute of Technology Karlsruhe Germany FZI Research Center for Information Technology Karlsruhe Germany

Reinforcement learning (RL) is a powerful approach for acquiring a good-performing policy. However, learning diverse skills is challenging in RL due to the commonly used Gaussian policy parameterization. We propose Diverse Skill learning (Di-SkilL1), an RL method for learning diverse skills using Mixture of Experts, where each expert formalizes a skill as a contextual motion primitive. Di-SkilL optimizes each expert and its associate context distribution to a maximum entropy objective that incentivizes learning diverse skills in similar contexts. The per-expert context distribution enables automatic curricula learning, allowing each expert to focus on its best-performing sub-region of the context space. To overcome hard discontinuities and multi-modalities without any prior knowledge of the environment’s unknown context probability space, we leverage energy-based models to represent the per-expert context distributions and demonstrate how we can efficiently train them using the standard policy gradient objective. We show on challenging robot simulation tasks that Di-SkilL can learn diverse and performant skills. © 2024, CC BY.

关键词： Reinforcement learning

Curriculum-Based Imitation of Versatile Skills

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Li, Maximilian Xiling Celik, Onur Becker, Philipp Blessing, Denis Lioutikov, Rudolf Neumann, Gerhard Intuitive Robots Lab Karlsruhe Institute of Technology Karlsruhe Germany Autonomous Learning Robots Karlsruhe Institute of Technology Karlsruhe Germany

learning skills by imitation is a promising concept for the intuitive teaching of robots. A common way to learn such skills is to learn a parametric model by maximizing the likelihood given the demonstrations. Yet, human demonstrations are often multi-modal, i.e., the same task is solved in multiple ways which is a major challenge for most imitation learning methods that are based on such a maximum likelihood (ML) objective. The ML objective forces the model to cover all data, it prevents specialization in the context space and can cause mode-averaging in the behavior space, leading to suboptimal or potentially catastrophic behavior. Here, we alleviate those issues by introducing a curriculum using a weight for each data point, allowing the model to specialize on data it can represent while incentivizing it to cover as much data as possible by an entropy bonus. We extend our algorithm to a Mixture of (linear) Experts (MoE) such that the single components can specialize on local context regions, while the MoE covers all data points. We evaluate our approach in complex simulated and real robot control tasks and show it learns from versatile human demonstrations and significantly outperforms current SOTA methods. © 2023, CC BY-NC-SA.

关键词： Curricula

Variational Distillation of Diffusion Policies into Mixture of Experts

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhou, Hongyi Blessing, Denis Li, Ge Celik, Onur Jia, Xiaogang Neumann, Gerhard Lioutikov, Rudolf Intuitive Robots Lab Karlsruhe Institute of Technology Germany Autonomous Learning Robots Karlsruhe Institute of Technology Germany FZI Research Center for Information Technology Germany

This work introduces Variational Diffusion Distillation (VDD), a novel method that distills denoising diffusion policies into Mixtures of Experts (MoE) through variational inference. Diffusion Models are the current state-of-the-art in generative modeling due to their exceptional ability to accurately learn and represent complex, multi-modal distributions. This ability allows Diffusion Models to replicate the inherent diversity in human behavior, making them the preferred models in behavior learning such as learning from Human Demonstrations (LfD). However, diffusion models come with some drawbacks, including the intractability of likelihoods and long inference times due to their iterative sampling process. The inference times, in particular, pose a significant challenge to real-time applications such as robot control. In contrast, MoEs effectively address the aforementioned issues while retaining the ability to represent complex distributions but are notoriously difficult to train. VDD is the first method that distills pre-trained diffusion models into MoE models, and hence, combines the expressiveness of Diffusion Models with the benefits of Mixture Models. Specifically, VDD leverages a decompositional upper bound of the variational objective that allows the training of each expert separately, resulting in a robust optimization scheme for MoEs. VDD demonstrates across nine complex behavior learning tasks, that it is able to: i) accurately distill complex distributions learned by the diffusion model, ii) outperform existing state-of-the-art distillation methods, and iii) surpass conventional methods for training MoE. The code and videos are available at https://***/vdd-website. © 2024, CC BY-NC-ND.

关键词： Variational techniques

GEOMETRY-AWARE RL FOR MANIPULATION OF VARYING SHAPES AND DEFORMABLE OBJECTS

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Hoang, Tai Le, Huy Becker, Philipp Vien, Ngo Anh Neumann, Gerhard Autonomous Learning Robots Karlsruhe Institute of Technology Bangladesh Bosch Center for Artificial Intelligence Bangladesh

Manipulating objects with varying geometries and deformable objects is a major challenge in robotics. Tasks such as insertion with different objects or cloth hanging require precise control and effective modelling of complex dynamics. In this work, we frame this problem through the lens of a heterogeneous graph that comprises smaller sub-graphs, such as actuators and objects, accompanied by different edge types describing their interactions. This graph representation serves as a unified structure for both rigid and deformable objects tasks, and can be extended further to tasks comprising multiple actuators. To evaluate this setup, we present a novel and challenging reinforcement learning benchmark, including rigid insertion of diverse objects, as well as rope and cloth manipulation with multiple end-effectors. These tasks present a large search space, as both the initial and target configurations are uniformly sampled in 3D space. To address this issue, we propose a novel graph-based policy model, dubbed Heterogeneous Equivariant Policy (HEPi), utilizing SE(3) equivariant message passing networks as the main backbone to exploit the geometric symmetry. In addition, by modeling explicit heterogeneity, HEPi can outperform Transformer-based and non-heterogeneous equivariant policies in terms of average returns, sample efficiency, and generalization to unseen objects. Our project page is available here. © 2025, CC BY.

关键词： End effectors

Curriculum-Based Imitation of Versatile Skills

学校读者我要写书评

暂无评论

Curriculum-Based Imitation of Versatile Skills

IEEE International Conference on Robotics and Automation (ICRA)

作者： Maximilian Xiling Li Onur Celik Philipp Becker Denis Blessing Rudolf Lioutikov Gerhard Neumann Intuitive Robots Lab Karlsruhe Institute of Technology Karlsruhe Germany Autonomous Learning Robots Karlsruhe Institute of Technology Karlsruhe Germany

关键词：

Swarm reinforcement learning for adaptive mesh refinement 23

学校读者我要写书评

暂无评论

Swarm reinforcement learning for adaptive mesh refinement

Proceedings of the 37th International Conference on Neural Information Processing Systems

作者： Niklas Freymuth Philipp Dahlinger Tobias Würth Simon Reisch Luise Kärger Gerhard Neumann Autonomous Learning Robots Karlsruhe Institute of Technology Karlsruhe Institute of Vehicle Systems Technology Karlsruhe Institute of Technology Karlsruhe

Adaptive Mesh Refinement (AMR) enhances the Finite Element Method, an important technique for simulating complex problems in engineering, by dynamically refining mesh regions, enabling a favorable trade-off between computational speed and simulation accuracy. Classical methods for AMR depend on heuristics or expensive error estimators, hindering their use for complex simulations. Recent learning-based AMR methods tackle these issues, but so far scale only to simple toy examples. We formulate AMR as a novel Adaptive Swarm Markov Decision Process in which a mesh is modeled as a system of simple collaborating agents that may split into multiple new agents. This framework allows for a spatial reward formulation that simplifies the credit assignment problem, which we combine with Message Passing Networks to propagate information between neighboring mesh elements. We experimentally validate our approach, Adaptive Swarm Mesh Refinement (ASMR), on challenging refinement tasks. Our approach learns reliable and efficient refinement strategies that can robustly generalize to different domains during inference. Additionally, it achieves a speedup of up to 2 orders of magnitude compared to uniform refinements in more demanding simulations. We outperform learned baselines and heuristics, achieving a refinement quality that is on par with costly error-based oracle AMR strategies.

关键词：