检索结果-内蒙古大学图书馆

Safe Multi-Agent Reinforcement learning for Behavior-Based Cooperative Navigation

学校读者我要写书评

暂无评论

IEEE robotics and Automation Letters 2025年第6期10卷 6256-6263页

作者： Dawood, Murad Pan, Sicong Dengler, Nils Zhou, Siqi Schoellig, Angela P. Bennewitz, Maren University of Bonn Humanoid Robots Lab Germany Center for Robotics Lamarr Institute for Machine Learning and Artificial Intelligence Bonn Germany The Technical University of Munich Learning Systems and Robotics lab Germany

In this paper, we address the problem of behavior-based cooperative navigation of mobile robots using safe multi-agent reinforcement learning (MARL). Our work is the first to focus on cooperative navigation without individual reference targets for the robots, using a single target for the formation's centroid. This eliminates the complexities involved in having several path planners to control a team of robots. To ensure safety, our MARL framework uses model predictive control (MPC) to prevent actions that could lead to collisions during training and execution. We demonstrate the effectiveness of our method in simulation and on real robots, achieving safe behavior-based cooperative navigation without using individual reference targets, with zero collisions, and faster target reaching compared to baselines. Finally, we study the impact of MPC safety filters on the learning process, revealing that we achieve faster convergence during training and we show that our approach can be safely deployed on real robots, even during early stages of the training. © 2016 IEEE.

关键词： Adversarial machine learning

Person Segmentation and Action Classification for Multi-Channel Hemisphere Field of View LiDAR Sensors

学校读者我要写书评

暂无评论

Person Segmentation and Action Classification for Multi-Chan...

2025 IEEE/SICE International Symposium on System Integration, SII 2025

作者： Seliunina, Svetlana Otelepko, Artem Memmesheimer, Raphael Behnke, Sven University of Bonn Autonomous Intelligent Systems Group Computer Science Institute VI - Intelligent Systems and Robotics Lamarr Institute for Machine Learning and Artificial Intelligence Center for Robotics Germany

ISBN: (纸本)9798331531614

Robots need to perceive persons in their surroundings for safety and to interact with them. In this paper, we present a person segmentation and action classification approach that operates on 3D scans of hemisphere field of view LiDAR sensors. We recorded a data set with an Ouster OSDome-64 sensor consisting of scenes where persons perform three different actions and annotated it. We propose a method based on a MaskDINO model to detect and segment persons and to recognize their actions from combined spherical projected multi-channel representations of the LiDAR data with an additional positional encoding. Our approach demonstrates good performance for the person segmentation task and further performs well for the estimation of the person action states walking, waving, and sitting. An ablation study provides insights about the individual channel contributions for the person segmentation task. The trained models, code and dataset are made publicly available. © 2025 IEEE.

关键词： Signal encoding

Epipolar Attention Field Transformers for Bird's Eye View Semantic Segmentation

学校读者我要写书评

暂无评论

Epipolar Attention Field Transformers for Bird's Eye View Se...

2025 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2025

作者： Witte, Christian Behley, Jens Stachniss, Cyrill Raaijmakers, Marvin Cariad Se Germany University of Bonn Center for Robotics Germany Lamarr Institute for Machine Learning and Artificial Intelligence Germany

ISBN: (纸本)9798331510831

Spatial understanding of the semantics of the surroundings is a key capability needed by autonomous cars to enable safe driving decisions. Recently, purely vision-based solutions have gained increasing research interest. In particular, approaches extracting a bird's eye view (BEV) from multiple cameras have demonstrated great performance for spatial understanding. This paper addresses the dependency on learned positional encodings to correlate image and BEV feature map elements for transformer-based methods. We propose leveraging epipolar geometric constraints to model the relationship between cameras and the BEV by Epipolar Attention Fields. They are incorporated into the attention mechanism as a novel attribution term, serving as an alternative to learned positional encodings. Experiments show that our method EAFormer outperforms previous BEV approaches by 2% mIoU for map semantic segmentation and exhibits superior generalization capabilities compared to implicitly learning the camera configuration. © 2025 IEEE.

关键词： Semantic Segmentation

RoboCup@Home 2024 OPL Winner NimbRo: Anthropomorphic Service Robots Using Foundation Models for Perception and Planning

学校读者我要写书评

暂无评论

RoboCup@Home 2024 OPL Winner NimbRo: Anthropomorphic Service...

27th RoboCup International Symposium, 2024

作者： Memmesheimer, Raphael Nogga, Jan Pätzold, Bastian Kruzhkov, Evgenii Bultmann, Simon Schreiber, Michael Bode, Jonas Karacora, Bertan Park, Juhui Savinykh, Alena Behnke, Sven Autonomous Intelligent Systems Computer Science Institute VI Lamarr Institute for Machine Learning and Artificial Intelligence and Center for Robotics University of Bonn Bonn Germany

ISBN: (纸本)9783031858581

We present the approaches and contributions of the winning team NimbRo@Home at the RoboCup@Home 2024 competition in the Open Platform League held in Eindhoven, NL. Further, we describe our hardware setup and give an overview of the results for the task stages and the final demonstration. For this year’s competition, we put a special emphasis on open-vocabulary object segmentation and grasping approaches that overcome the labeling overhead of supervised vision approaches, commonly used in RoboCup@Home. We successfully demonstrated that we can segment and grasp non-labeled objects by text descriptions. Further, we extensively employed Large Language Model (LLMs) for natural language understanding and task planning. Throughout the competition, our approaches showed robustness and generalization capabilities. A video of our performance can be found online (https://***/videos/RoboCup_2024). © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

关键词： Robot programming

Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attacks

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Hu, Hanjiang Robey, Alexander Liu, Changliu Robotics Institute Machine Learning Department Carnegie Mellon University United States Machine Learning Department Carnegie Mellon University United States Robotics Institute Carnegie Mellon University United States

Large language models (LLMs) are highly vulnerable to jailbreaking attacks, wherein adversarial prompts are designed to elicit harmful responses. While existing defenses effectively mitigate single-turn attacks by detecting and filtering unsafe inputs, they fail against multi-turn jailbreaks that exploit contextual drift over multiple interactions, gradually leading LLMs away from safe behavior. To address this challenge, we propose a safety steering framework grounded in safe control theory, ensuring invariant safety in multi-turn dialogues. Our approach models the dialogue with LLMs using state-space representations and introduces a novel neural barrier function (NBF) to detect and filter harmful queries emerging from evolving contexts proactively. Our method achieves invariant safety at each turn of dialogue by learning a safety predictor that accounts for adversarial queries, preventing potential context drift toward jailbreaks. Extensive experiments under multiple LLMs show that our NBF-based safety steering outperforms safety alignment baselines, offering stronger defenses against multi-turn jailbreaks while maintaining a better trade-off between safety and helpfulness under different multi-turn jailbreak methods. © 2025, CC BY-NC-SA.

关键词： Invariance

GCE-Pose: Global Context Enhancement for Category-level Object Pose Estimation

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Li, Weihang Xu, Hongli Huang, Junwen Jung, Hyunjun Yu, Peter K.T. Navab, Nassir Busam, Benjamin Technical University of Munich Germany Munich Center for Machine Learning Germany XYZ Robotics

A key challenge in model-free category-level pose estimation is the extraction of contextual object features that generalize across varying instances within a specific category. Recent approaches leverage foundational features to capture semantic and geometry cues from data. However, these approaches fail under partial visibility. We overcome this with a first-complete-then-aggregate strategy for feature extraction utilizing class priors. In this paper, we present GCE-Pose, a method that enhances pose estimation for novel instances by integrating category-level global context prior. GCE-Pose performs semantic shape reconstruction with a proposed Semantic Shape Reconstruction (SSR) module. Given an unseen partial RGB-D object instance, our SSR module reconstructs the instance’s global geometry and semantics by deforming category-specific 3D semantic prototypes through a learned deep Linear Shape Model. We further introduce a Global Context Enhanced (GCE) feature fusion module that effectively fuses features from partial RGB-D observations and the reconstructed global context. Extensive experiments validate the impact of our global context prior and the effectiveness of the GCE fusion module, demonstrating that GCE-Pose significantly outperforms existing methods on challenging real-world datasets HouseCat6D and NOCS-REAL275. Our project page is available at https://***/GCEPose/. Copyright © 2025, The Authors. All rights reserved.

关键词： Feature extraction

PlaySlot: learning Inverse Latent Dynamics for Controllable Object-Centric Video Prediction and Planning

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Villar-Corrales, Angel Behnke, Sven Computer Science Institute VI – Intelligent Systems and Robotics Center for Robotics The Lamarr Institute for Machine Learning and Artificial Intelligence Germany

Predicting future scene representations is a crucial task for enabling robots to understand and interact with the environment. However, most existing methods rely on video sequences and simulations with precise action annotations, limiting their ability to leverage the large amount of available unlabeled video data. To address this challenge, we propose PlaySlot, an object-centric video prediction model that infers object representations and latent actions from unlabeled video sequences. It then uses these representations to forecast future object states and video frames. PlaySlot allows to generate multiple possible futures conditioned on latent actions, which can be inferred from video dynamics, provided by a user, or generated by a learned action policy, thus enabling versatile and interpretable world modeling. Our results show that PlaySlot outperforms both stochastic and object-centric baselines for video prediction across different environments. Furthermore, we show that our inferred latent actions can be used to learn robot behaviors sample-efficiently from unlabeled video demonstrations. Videos and code are available at https://***/PlaySlot/. © 2025, CC BY.

关键词： Stochastic systems

3D Hierarchical Panoptic Segmentation in Real Orchard Environments Across Different Sensors

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Sodano, Matteo Magistri, Federico Marks, Elias Hosn, Fares Zurbayev, Aibek Marcuzzi, Rodrigo Malladi, Meher V.R. Behley, Jens Stachniss, Cyrill Center for Robotics University of Bonn Germany Lamarr Institute for Machine Learning and Artificial Intelligence Germany

Crop yield estimation is a relevant problem in agriculture, because an accurate crop yield estimate can support farmers’ decisions on harvesting or precision intervention. Robots can help to automate this process. To do so, they need to be able to perceive the surrounding environment to identify target objects. In this paper, we introduce a novel approach to address the problem of hierarchical panoptic segmentation of apple orchards on 3D data from different sensors. Our approach is able to simultaneously provide semantic segmentation, instance segmentation of trunks and fruits, and instance segmentation of plants (a single trunk with its fruits). This allows us to identify relevant information such as individual plants, fruits, and trunks, and capture the relationship among them, such as precisely estimate the number of fruits associated to each tree in an orchard. Additionally, to efficiently evaluate our approach for hierarchical panoptic segmentation, we provide a dataset designed specifically for this task. Our dataset is recorded in Bonn in a real apple orchard with a variety of sensors, spanning from a terrestrial laser scanner to a RGB-D camera mounted on different robotic platforms. The experiments show that our approach surpasses state-of-the-art approaches in 3D panoptic segmentation in the agricultural domain, while also providing full hierarchical panoptic segmentation. Our dataset has been made publicly available at https://***/data/hops/. We will provide the open-source implementation of our approach and public competiton for hierarchical panoptic segmentation on the hidden test sets upon paper acceptance. Copyright © 2025, The Authors. All rights reserved.

关键词： Fruits

Towards Generating Realistic 3D Semantic Training Data for Autonomous Driving

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Nunes, Lucas Marcuzzi, Rodrigo Behley, Jens Stachniss, Cyrill Center for Robotics University of Bonn Germany Lamarr Institute for Machine Learning and Artificial Intelligence Germany

Semantic scene understanding is crucial for robotics and computer vision applications. In autonomous driving, 3D semantic segmentation plays an important role for enabling safe navigation. Despite significant advances in the field, the complexity of collecting and annotating 3D data is a bottleneck in this developments. To overcome that data annotation limitation, synthetic simulated data has been used to generate annotated data on demand. There is still however a domain gap between real and simulated data. More recently, diffusion models have been in the spotlight, enabling close-to-real data synthesis. Those generative models have been recently applied to the 3D data domain for generating scene-scale data with semantic annotations. Still, those methods either rely on image projection or decoupled models trained with different resolutions in a coarse-to-fine manner. Such intermediary representations impact the generated data quality due to errors added in those transformations. In this work, we propose a novel approach able to generate 3D semantic scene-scale data without relying on any projection or decoupled trained multi-resolution models, achieving more realistic semantic scene data generation compared to previous state-of-the-art methods. Besides improving 3D semantic scene-scale data synthesis, we thoroughly evaluate the use of the synthetic scene samples as labeled data to train a semantic segmentation network. In our experiments, we show that using the synthetic annotated data generated by our method as training data together with the real semantic segmentation labels, leads to an improvement in the semantic segmentation model performance. Our results show the potential of generated scene-scale point clouds to generate more training data to extend existing datasets, reducing the data annotation effort. Our code is available at https://***/PRBonn/3DiSS. Copyright © 2025, The Authors. All rights reserved.

关键词： Semantic Segmentation