检索结果-内蒙古大学图书馆

Tracking Control for a Spherical Pendulum via Curriculum Reinforcement learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Klink, Pascal Wolf, Florian Ploeger, Kai Peters, Jan Pajarinen, Joni The Intelligent Autonomous Systems Group the Technical University of Darmstadt Germany The Department of Electrical Engineering and Automation Aalto University Finland The German Research Center for AI Research Department Systems AI for Robot Learning Hessian.AI the Centre of Cognitive Science Germany

Reinforcement learning (RL) allows learning nontrivial robot control laws purely from data. However, many successful applications of RL have relied on ad-hoc regularizations, such as hand-crafted curricula, to regularize the learning performance. In this paper, we pair a recent algorithm for automatically building curricula with RL on massively parallelized simulations to learn a tracking controller for a spherical pendulum on a robotic arm via RL. Through an improved optimization scheme that better respects the non-Euclidean task structure, we allow the method to reliably generate curricula of trajectories to be tracked, resulting in faster and more robust learning compared to an RL baseline that does not exploit this form of structured learning. The learned policy matches the performance of an optimal control baseline on the real system, demonstrating the potential of curriculum RL to jointly learn state estimation and control for non-linear tracking tasks. Copyright © 2023, The Authors. All rights reserved.

关键词： Reinforcement learning

Global Tensor Motion Planning

学校读者我要写书评

暂无评论

IEEE robotics and Automation Letters 2025年第7期10卷 7302-7309页

作者： An T. Le Kay Pompetzki João Carvalho Joe Watson Julen Urain Armin Biess Georgia Chalvatzaki Jan Peters Intelligent Autonomous Systems Lab TU Darmstadt Darmstadt Germany German Research Center for AI (DFKI) Kaiserslautern Germany Interactive Robot Perception & Learning Lab TU Darmstadt Darmstadt Germany Hessian.AI Darmstadt Germany Centre for Cognitive Science Darmstadt Germany

Batch planning is increasingly necessary to quickly produce diverse and quality motion plans for downstream learning applications, such as distillation and imitation learning. This letter presents Global Tensor Motion Planning (GTMP)—a sampling-based motion planning algorithm comprising only tensor operations. We introduce a novel discretization structure represented as a random multipartite graph, enabling efficient vectorized sampling, collision checking, and search. We provide a theoretical investigation showing that GTMP exhibits probabilistic completeness while supporting modern GPU/TPU. Additionally, by incorporating smooth structures into the multipartite graph, GTMP directly plans smooth splines without requiring gradient-based optimization. Experiments on lidar-scanned occupancy maps and the MotionBenchMarker dataset demonstrate GTMP's computation efficiency in batch planning compared to baselines, underscoring GTMP's potential as a robust, scalable planner for diverse applications and large-scale robot learning tasks.

关键词： Planning Tensors Costs Splines (mathematics) Probabilistic logic Time complexity Cost function Complexity theory Artificial intelligence Training

Safe Reinforcement learning on the Constraint Manifold: Theory and Applications

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Liu, Puze Bou-Ammar, Haitham Peters, Jan Tateo, Davide Intelligent Autonomous Systems Group The Technical University of Darmstadt Germany Huawei R&D London United Kingdom Department of Systems AI for Robot Learning German Research Center for AI Germany Hessian Centre for Artificial Intelligence The Centre of Cognitive Science

Integrating learning-based techniques, especially reinforcement learning, into robotics is promising for solving complex problems in unstructured environments. However, most existing approaches are trained in well-tuned simulators and subsequently deployed on real robots without online fine-tuning. In this setting, extensive engineering is required to mitigate the sim-to-real gap, which can be challenging for complex systems. Instead, learning with real-world interaction data offers a promising alternative: it not only eliminates the need for a fine-tuned simulator but also applies to a broader range of tasks where accurate modeling is unfeasible. One major problem for on-robot reinforcement learning is ensuring safety, as uncontrolled exploration can cause catastrophic damage to the robot or the environment. Indeed, safety specifications, often represented as constraints, can be complex and non-linear, making safety challenging to guarantee in learning systems. In this paper, we show how we can impose complex safety constraints on learning-based robotics systems in a principled manner, both from theoretical and practical points of view. Our approach is based on the concept of the Constraint Manifold, representing the set of safe robot configurations. Exploiting differential geometry techniques, i.e., the tangent space, we can construct a safe action space, allowing learning agents to sample arbitrary actions while ensuring safety. We demonstrate the method’s effectiveness in a real-world robot air Hockey task, showing that our method can handle high-dimensional tasks with complex constraints. Videos of the real robot experiments are available publicly1 Copyright © 2024, The Authors. All rights reserved.

关键词： Reinforcement learning

Zero-Shot Transfer of a Tactile-based Continuous Force Control Policy from Simulation to robot

学校读者我要写书评

暂无评论

Zero-Shot Transfer of a Tactile-based Continuous Force Contr...

IEEE/RSJ International Conference on Intelligent robots and systems (IROS)

作者： Luca Lach Robert Haschke Davide Tateo Jan Peters Helge Ritter Júlia Borràs Carme Torras Institut de Robòtica i Informàtica Industrial CSIC-UPC Neuroinformatics Group Technical Faculty Bielefeld University Computer Science Department Technical University Darmstadt German Research Center for AI (DFKI) Research Department: Systems AI for Robot Learning Hessian.AI Centre for Cognitive Science

ISBN: (数字)9798350377705

ISBN: (纸本)9798350377712

The advent of tactile sensors in robotics has sparked many ideas on how robots can leverage direct contact measurements of their environment interactions to improve manipulation tasks. An important line of research in this regard is grasp force control, which aims to manipulate objects safely by limiting the amount of force exerted on the object. While prior works have either hand-modeled their force controllers, employed model-based approaches, or not shown sim-to-real transfer, we propose a model-free deep reinforcement learning approach trained in simulation and then transferred to the robot without further fine-tuning. We, therefore, present a simulation environment that produces realistic normal forces, which we use to train continuous force control policies. A detailed evaluation shows that the learned policy performs similarly or better than a hand-crafted baseline. Ablation studies prove that the proposed inductive bias and domain randomization facilitate sim-to-real transfer. Code, models, and supplementary videos are available on https://***/view/rl-force-ctrl

关键词： Limiting Shape Force Tactile sensors Deep reinforcement learning Force control Grippers robots Intelligent robots Videos

One Policy to Run Them All: an End-to-end learning Approach to Multi-Embodiment Locomotion

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Bohlinger, Nico Czechmanowski, Grzegorz Krupka, Maciej Kicki, Piotr Walas, Krzysztof Peters, Jan Tateo, Davide Department of Computer Science Technical University of Darmstadt Germany Institute of Robotics and Machine Intelligence Poznan University of Technology Poland Research Department: Systems AI for Robot Learning Germany IDEAS NCBR Warsaw Poland Hessian.AI Centre for Cognitive Science

Deep Reinforcement learning techniques are achieving state-of-the-art results in robust legged locomotion. While there exists a wide variety of legged platforms such as quadruped, humanoids, and hexapods, the field is still missing a single learning framework that can control all these different embodiments easily and effectively and possibly transfer, zero or few-shot, to unseen robot embodiments. We introduce URMA, the Unified robot Morphology Architecture, to close this gap. Our framework brings the end-to-end Multi-Task Reinforcement learning approach to the realm of legged robots, enabling the learned policy to control any type of robot morphology. The key idea of our method is to allow the network to learn an abstract locomotion controller that can be seamlessly shared between embodiments thanks to our morphology-agnostic encoders and decoders. This flexible architecture can be seen as a potential first step in building a foundation model for legged robot locomotion. Our experiments show that URMA can learn a locomotion policy on multiple embodiments that can be easily transferred to unseen robot platforms in simulation and the real world. Copyright © 2024, The Authors. All rights reserved.

关键词： Reinforcement learning

Adapting Object-Centric Probabilistic Movement Primitives with Residual Reinforcement learning

学校读者我要写书评

暂无评论

Adapting Object-Centric Probabilistic Movement Primitives wi...

IEEE-RAS International Conference on Humanoid robots

作者： João Carvalho Dorothea Koert Marek Daniv Jan Peters Computer Science Department TU Darmstadt (TUDa) Institute for Intelligent Autonomous Systems TUDa Centre for Cognitive Science Research Department: Systems AI for Robot Learning German Research Center for AI (DFKI) Hessian. AI

ISBN: (纸本)9798350309805

It is desirable for future robots to quickly learn new tasks and adapt learned skills to constantly changing environments. To this end, Probabilistic Movement Primitives (ProMPs) have shown to be a promising framework to learn generalizable trajectory generators from distributions over demonstrated trajectories. However, in practical applications that require high precision in the manipulation of objects, the accuracy of ProMPs is often insufficient, in particular when they are learned in cartesian space from external observations and executed with limited controller gains. Therefore, we propose to combine ProMPs with the Residual Reinforcement learning (RRL) framework, to account for both, corrections in position and orientation during task execution. In particular, we learn a residual on top of a nominal ProMP trajectory with Soft Actor-Critic and incorporate the variability in the demonstrations as a decision variable to reduce the search space for RRL. As a proof of concept, we evaluate our proposed method on a 3D block insertion task with a 7-DoF Franka Emika Panda robot. Experimental results show that the robot successfully learns to complete the insertion, which was not possible before with using basic ProMPs.

关键词： Three-dimensional displays Humanoid robots Reinforcement learning Aerospace electronics Probabilistic logic Generators Trajectory

Safe Reinforcement learning of Dynamic High-Dimensional robotic Tasks: Navigation, Manipulation, Interaction

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Liu, Puze Zhang, Kuo Tateo, Davide Jauhri, Snehal Hu, Zhiyuan Peters, Jan Chalvatzaki, Georgia Computer Science Department Technical University Darmstadt Germany Research Department: Systems AI for Robot Learning Germany Hessian.AI Centre for Cognitive Science

Safety is a fundamental property for the real-world deployment of robotic platforms. Any control policy should avoid dangerous actions that could harm the environment, humans, or the robot itself. In reinforcement learning (RL), safety is crucial when exploring a new environment to learn a new skill. This paper introduces a new formulation of safe exploration for robot Reinforcement learning in the tangent space of the constraint manifold that effectively transforms the action space of the RL agent for always respecting safety constraints locally. We show how to apply this approach to a wide range of robotic platforms and how to define safety constraints that represent dynamic articulated objects like humans in the context of robotic RL. Our proposed approach achieves state-of-the-art performance in simulated high-dimensional and dynamic tasks while avoiding collisions with the environment. We show safe real-world deployment of our learned controller on a TIAGo++ robot, achieving remarkable performance in manipulation and human-robot interaction tasks. Copyright © 2022, The Authors. All rights reserved.

关键词： Reinforcement learning

DIMINISHING RETURN OF VALUE EXPANSION METHODS IN MODEL-BASED REINFORCEMENT learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Palenicek, Daniel Lutter, Michael Carvalho, João Peters, Jan Intelligent Autonomous Systems Technical University of Darmstadt Germany Hessian.AI Hochschulstr. 10 Darmstadt64293 Germany Research Department: Systems AI for Robot Learning Germany Centre for Cognitive Science Hochschulstr. 10 Darmstadt64293 Germany

Model-based reinforcement learning is one approach to increase sample efficiency. However, the accuracy of the dynamics model and the resulting compounding error over modelled trajectories are commonly regarded as key limitations. A natural question to ask is: How much more sample efficiency can be gained by improving the learned dynamics models? Our paper empirically answers this question for the class of model-based value expansion methods in continuous control problems. Value expansion methods should benefit from increased model accuracy by enabling longer rollout horizons and better value function approximations. Our empirical study, which leverages oracle dynamics models to avoid compounding model errors, shows that (1) longer horizons increase sample efficiency, but the gain in improvement decreases with each additional expansion step, and (2) the increased model accuracy only marginally increases the sample efficiency compared to learned models with identical horizons. Therefore, longer horizons and increased model accuracy yield diminishing returns in terms of sample efficiency. These improvements in sample efficiency are particularly disappointing when compared to model-free value expansion methods. Even though they introduce no computational overhead, we find their performance to be on-par with model-based value expansion methods. Therefore, we conclude that the limitation of model-based value expansion methods is not the model accuracy of the learned models. While higher model accuracy is beneficial, our experiments show that even a perfect model will not provide an un-rivalled sample efficiency but that the bottleneck lies elsewhere. Copyright © 2023, The Authors. All rights reserved.

关键词： Efficiency

Placing by Touching: An Empirical Study on the Importance of Tactile Sensing for Precise Object Placing

学校读者我要写书评

暂无评论

Placing by Touching: An Empirical Study on the Importance of...

IEEE/RSJ International Conference on Intelligent robots and systems (IROS)

作者： Luca Lach Niklas Funk Robert Haschke Séverin Lemaignan Helge Joachim Ritter Jan Peters Georgia Chalvatzaki Neuroinformatics Group Technical Faculty Bielefeld University Institut de Robòtica i Informatica Industrial CSIC- UPC Computer Science Department Technical University Darmstadt PAL Robotics Research Department: Systems AI for Robot Learning German Research Center for AI (DFKI) Hessian AI Centre for Cognitive Science

This work deals with a practical everyday problem: stable object placement on flat surfaces starting from unknown initial poses. Common object-placing approaches require either complete scene specifications or extrinsic sensor measurements, e.g., cameras, that occasionally suffer from occlusions. We propose a novel approach for stable object placing that combines tactile feedback and proprioceptive sensing. We devise a neural architecture called PlaceNet that estimates a rotation matrix, resulting in a corrective gripper movement that aligns the object with the placing surface for the subsequent object manipulation. We compare models with different sensing modalities, such as force-torque, an external motion capture system, and two classical baseline models in real-world object placing tasks with different objects. The experimental evaluation of our placing policies with a set of unseen everyday objects reveals significant generalization of our proposed pipeline, suggesting that tactile sensing plays a vital role in the intrinsic understanding of robotic dexterous object manipulation. Code, models, and supplementary videos are available on https://***/view/placing-by-touching.

关键词：