检索结果-内蒙古大学图书馆

arXiv 2022年

作者： Becker, Philipp Neumann, Gerhard Autonomous Learning Robots Lab Karlsruhe Institute of Technology Germany

Improved state space models, such as Recurrent State Space Models (RSSMs), are a key factor behind recent advances in model-based reinforcement learning (RL). Yet, despite their empirical success, many of the underlying design choices are not well understood. We show that RSSMs use a suboptimal inference scheme and that models trained using this inference overestimate the aleatoric uncertainty of the ground truth system. We find this overestimation implicitly regularizes RSSMs and allows them to succeed in model-based RL. We postulate that this implicit regularization fulfills the same functionality as explicitly modeling epistemic uncertainty, which is crucial for many other model-based RL approaches. Yet, overestimating aleatoric uncertainty can also impair performance in cases where accurately estimating it matters, e.g., when we have to deal with occlusions, missing observations, or fusing sensor modalities at different frequencies. Moreover, the implicit regularization is a side-effect of the inference scheme and not the result of a rigorous, principled formulation, which renders analyzing or improving RSSMs difficult. Thus, we propose an alternative approach building on well-understood components for modeling aleatoric and epistemic uncertainty, dubbed Variational Recurrent Kalman Network (VRKN). This approach uses Kalman updates for exact smoothing inference in a latent space and Monte Carlo Dropout to model epistemic uncertainty. Due to the Kalman updates, the VRKN can naturally handle missing observations or sensor fusion problems with varying numbers of observations per time step. Our experiments show that using the VRKN instead of the RSSM improves performance in tasks where appropriately capturing aleatoric uncertainty is crucial while matching it in the deterministic standard benchmarks1 © 2022, CC BY.

关键词： State space methods

来源：评论

学校读者我要写书评

暂无评论

Latent Task-Specific Graph Network Simulators

arXiv

引用

arXiv 2023年

作者： Dahlinger, Philipp Freymuth, Niklas Volpp, Michael Hoang, Tai Neumann, Gerhard Autonomous Learning Robots Karlsruhe Institute of Technology Karlsruhe Germany Bosch Center for Artificial Intelligence Renningen Germany

Simulating dynamic physical interactions is a critical challenge across multiple scientific domains, with applications ranging from robotics to material science. For mesh-based simulations, Graph Network Simulators (GNSs) pose an efficient alternative to traditional physics-based simulators. Their inherent differentiability and speed make them particularly well-suited for inverse design problems. Yet, adapting to new tasks from limited available data is an important aspect for real-world applications that current methods struggle with. We frame mesh-based simulation as a meta-learning problem and use a recent Bayesian meta-learning method to improve GNSs adaptability to new scenarios by leveraging context data and handling uncertainties. Our approach, latent task-specific graph network simulator, uses non-amortized task posterior approximations to sample latent descriptions of unknown system properties. Additionally, we leverage movement primitives for efficient full trajectory prediction, effectively addressing the issue of accumulating errors encountered by previous auto-regressive methods. We validate the effectiveness of our approach through various experiments, performing on par with or better than established baseline methods. Movement primitives further allow us to accommodate various types of context data, as demonstrated through the utilization of point clouds during inference. By combining GNSs with meta-learning, we bring them closer to real-world applicability, particularly in scenarios with smaller datasets. © 2023, CC BY-NC-SA.

关键词： Mesh generation

来源：评论

学校读者我要写书评

暂无评论

A Study on Dense and Sparse (Visual) Rewards in Robot Policy learning 22nd

A Study on Dense and Sparse (Visual) Rewards in Robot Policy...

引用

22th Annual Conference Towards autonomous Robotic Systems, TAROS 2021

作者： Mohtasib, Abdalkarim Neumann, Gerhard Cuayáhuitl, Heriberto Lincoln Centre for Autonomous Systems University of Lincoln Lincoln United Kingdom Autonomous Learning Robots Karlsruhe Institute of Technology Karlsruhe Germany

ISBN: (纸本)9783030891763

Deep Reinforcement learning (DRL) is a promising approach for teaching robots new behaviour. However, one of its main limitations is the need for carefully hand-coded reward signals by an expert. We argue that it is crucial to automate the reward learning process so that new skills can be taught to robots by their users. To address such automation, we consider task success classifiers using visual observations to estimate the rewards in terms of task success. In this work, we study the performance of multiple state-of-the-art deep reinforcement learning algorithms under different types of reward: Dense, Sparse, Visual Dense, and Visual Sparse rewards. Our experiments in various simulation tasks (Pendulum, Reacher, Pusher, and Fetch Reach) show that while DRL agents can learn successful behaviours using visual rewards when the goal targets are distinguishable, their performance may decrease if the task goal is not clearly visible. Our results also show that visual dense rewards are more successful than visual sparse rewards and that there is no single best algorithm for all tasks. © 2021, Springer Nature Switzerland AG.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

HIDDEN PARAMETER RECURRENT STATE SPACE MODELS FOR CHANGING DYNAMICS SCENARIOS 10

HIDDEN PARAMETER RECURRENT STATE SPACE MODELS FOR CHANGING D...

引用

10th International Conference on learning Representations, ICLR 2022

作者： Shaj, Vaisakh Büchler, Dieter Sonker, Rohit Becker, Philipp Neumann, Gerhard Autonomous Learning Robots KIT Germany LCAS University of Lincoln United Kingdom Max Planck Institute for Intelligent Systems Tübingen Germany Indian Institute of Technology Kanpur India

Recurrent State-space models (RSSMs) are highly expressive models for learning patterns in time series data and system identification. However, these models assume that the dynamics are fixed and unchanging, which is rarely the case in real-world scenarios. Many control applications often exhibit tasks with similar but not identical dynamics which can be modeled as a latent variable. We introduce the Hidden Parameter Recurrent State Space Models (HiP-RSSMs), a framework that parametrizes a family of related dynamical systems with a low-dimensional set of latent factors. We present a simple and effective way of learning and performing inference over this Gaussian graphical model that avoids approximations like variational inference. We show that HiP-RSSMs outperforms RSSMs and competing multi-task models on several challenging robotic benchmarks both on real-world systems and simulations. © 2022 ICLR 2022 - 10th International Conference on learning Representationss. All rights reserved.

关键词： Dynamics

来源：评论

学校读者我要写书评

暂无评论

Category-Agnostic 6D Pose Estimation with Conditional Neural Processes

arXiv

引用

arXiv 2022年

作者： Li, Yumeng Gao, Ning Ziesche, Hanna Neumann, Gerhard Bosch Center for Artificial Intelligence Germany Autonomous Learning Robots KIT Germany

We present a novel meta-learning approach for 6D pose estimation on unknown objects. In contrast to "instance-level" and "category-level" pose estimation methods, our algorithm learns object representation in a category-agnostic way, which endows it with strong generalization capabilities across object categories. Specifically, we employ a neural process-based meta-learning approach to train an encoder to capture texture and geometry of an object in a latent representation, based on very few RGB-D images and ground-truth keypoints. The latent representation is then used by a simultaneously meta-trained decoder to predict the 6D pose of the object in new images. Furthermore, we propose a novel geometry-aware decoder for the keypoint prediction using a Graph Neural Network (GNN), which explicitly takes geometric constraints specific to each object into consideration. To evaluate our algorithm, extensive experiments are conducted on the LineMOD dataset, and on our new fully-annotated synthetic datasets generated from Multiple Categories in Multiple Scenes (MCMS). Experimental results demonstrate that our model performs well on unseen objects with very different shapes and appearances. Remarkably, our model also shows robust performance on occluded scenes although trained fully on data without occlusion. To our knowledge, this is the first work exploring cross-category level 6D pose estimation. © 2022, CC BY-NC-ND.

关键词： Geometry

来源：评论

学校读者我要写书评

暂无评论

Swarm reinforcement learning for adaptive mesh refinement 23

Swarm reinforcement learning for adaptive mesh refinement

引用

Proceedings of the 37th International Conference on Neural Information Processing Systems

作者： Niklas Freymuth Philipp Dahlinger Tobias Würth Simon Reisch Luise Kärger Gerhard Neumann Autonomous Learning Robots Karlsruhe Institute of Technology Karlsruhe Institute of Vehicle Systems Technology Karlsruhe Institute of Technology Karlsruhe

Adaptive Mesh Refinement (AMR) enhances the Finite Element Method, an important technique for simulating complex problems in engineering, by dynamically refining mesh regions, enabling a favorable trade-off between computational speed and simulation accuracy. Classical methods for AMR depend on heuristics or expensive error estimators, hindering their use for complex simulations. Recent learning-based AMR methods tackle these issues, but so far scale only to simple toy examples. We formulate AMR as a novel Adaptive Swarm Markov Decision Process in which a mesh is modeled as a system of simple collaborating agents that may split into multiple new agents. This framework allows for a spatial reward formulation that simplifies the credit assignment problem, which we combine with Message Passing Networks to propagate information between neighboring mesh elements. We experimentally validate our approach, Adaptive Swarm Mesh Refinement (ASMR), on challenging refinement tasks. Our approach learns reliable and efficient refinement strategies that can robustly generalize to different domains during inference. Additionally, it achieves a speedup of up to 2 orders of magnitude compared to uniform refinements in more demanding simulations. We outperform learned baselines and heuristics, achieving a refinement quality that is on par with costly error-based oracle AMR strategies.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Domain-Specific Fine-Tuning of Large Language Models for Interactive Robot Programming

arXiv

引用

arXiv 2023年

作者： Alt, Benjamin Keßner, Urs Taranovic, Aleksandar Katic, Darko Hermann, Andreas Jäkel, Rainer Neumann, Gerhard ArtiMinds Robotics Karlsruhe76131 Germany Karlsruhe Institute of Technology Autonomous Learning Robots Lab Karlsruhe76131 Germany

Industrial robots are applied in a widening range of industries, but robot programming mostly remains a task limited to programming experts. We propose a natural language-based assistant for programming of advanced, industrial robotic applications and investigate strategies for domain-specific fine-tuning of foundation models with limited data and *** Codes 68T40 © 2023, CC BY-SA.

关键词： Robot programming

来源：评论

学校读者我要写书评

暂无评论

Beyond ELBOs: A Large-Scale Evaluation of Variational Methods for Sampling

arXiv

引用

arXiv 2024年

作者： Blessing, Denis Jia, Xiaogang Esslinger, Johannes Vargas, Francisco Neumann, Gerhard Autonomous Learning Robots Karlsruhe Institute of Technology Karlsruhe Germany University of Cambridge Cambridge United Kingdom FZI Research Center for Information Technology Karlsruhe Germany

Monte Carlo methods, Variational Inference, and their combinations play a pivotal role in sampling from intractable probability distributions. However, current studies lack a unified evaluation framework, relying on disparate performance measures and limited method comparisons across diverse tasks, complicating the assessment of progress and hindering the decision-making of practitioners. In response to these challenges, our work introduces a benchmark that evaluates sampling methods using a standardized task suite and a broad range of performance criteria. Moreover, we study existing metrics for quantifying mode collapse and introduce novel metrics for this purpose. Our findings provide insights into strengths and weaknesses of existing sampling methods, serving as a valuable reference for future developments. The code is publicly available here. © 2024, CC BY.

关键词： Monte Carlo methods

来源：评论

学校读者我要写书评

暂无评论

Inferring Versatile Behavior from Demonstrations by Matching Geometric Descriptors

arXiv

引用

arXiv 2022年

作者： Freymuth, Niklas Jordi Schreiber, Nicolas Becker, Philipp Taranovic, Aleksandar Neumann, Gerhard Autonomous Learning Robots Karlsruhe Institute of Technology Karlsruhe Germany Renningen Germany

Humans intuitively solve tasks in versatile ways, varying their behavior in terms of trajectory-based planning and for individual steps. Thus, they can easily generalize and adapt to new and changing environments. Current Imitation learning algorithms often only consider unimodal expert demonstrations and act in a state-action-based setting, making it difficult for them to imitate human behavior in case of versatile demonstrations. Instead, we combine a mixture of movement primitives with a distribution matching objective to learn versatile behaviors that match the expert’s behavior and versatility. To facilitate generalization to novel task configurations, we do not directly match the agent’s and expert’s trajectory distributions but rather work with concise geometric descriptors which generalize well to unseen task configurations. We empirically validate our method on various robot tasks using versatile human demonstrations and compare to imitation learning algorithms in a state-action setting as well as a trajectory-based setting. We find that the geometric descriptors greatly help in generalizing to new task configurations and that combining them with our distribution-matching objective is crucial for representing and reproducing versatile behavior. © 2022, CC BY.

关键词： Demonstrations

来源：评论

学校读者我要写书评

暂无评论

learning Sub-Second Routing Optimization in Computer Networks requires Packet-Level Dynamics

arXiv

引用

arXiv 2024年

作者： Boltres, Andreas Freymuth, Niklas Jahnke, Patrick Karl, Holger Neumann, Gerhard Autonomous Learning Robots Karlsruhe Institute of Technology SAP SE Germany Turba AI United States Internet-Technology and Softwarization Hasso-Plattner-Institut Potsdam Germany

Finding efficient routes for data packets is an essential task in computer networking. The optimal routes depend greatly on the current network topology, state and traffic demand, and they can change within milliseconds. Reinforcement learning can help to learn network representations that provide routing decisions for possibly novel situations. So far, this has commonly been done using fluid network models. We investigate their suitability for millisecond-scale adaptations with a range of traffic mixes and find that packet-level network models are necessary to capture true dynamics, in particular in the presence of TCP traffic. To this end, we present PackeRL, the first packet-level Reinforcement learning environment for routing in generic network topologies. Our experiments confirm that learning-based strategies that have been trained in fluid environments do not generalize well to this more realistic, but more challenging setup. Hence, we also introduce two new algorithms for learning sub-second Routing Optimization. We present M-Slim, a dynamic shortest-path algorithm that excels at high traffic volumes but is computationally hard to scale to large network topologies, and FieldLines, a novel next-hop policy design that re-optimizes routing for any network topology within milliseconds without requiring any re-training. Both algorithms outperform current learning-based approaches as well as commonly used static baseline protocols in scenarios with high-traffic volumes. All findings are backed by extensive experiments in realistic network conditions in our fast and versatile training and evaluation framework. © 2024, CC BY.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：