检索结果-内蒙古大学图书馆

FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interaction

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Rotondi, Dennis Scaparro, Fabio Blum, Hermann Arras, Kai O. Socially Intelligent Robotics Lab Institute for Artificial Intelligence University of Stuttgart Germany Robot Perception and Learning Lab LAMARR Institute for Machine Learning and Artificial Intelligence University of Bonn Germany

The concept of 3D scene graphs is increasingly recognized as a powerful semantic and hierarchical representation of the environment. Current approaches often address this at a coarse, object-level resolution. In contrast, our goal is to develop a representation that enables robots to directly interact with their environment by identifying both the location of functional interactive elements and how these can be used. To achieve this, we focus on detecting and storing objects at a finer resolution, focusing on affordance-relevant parts. The primary challenge lies in the scarcity of data that extends beyond instance-level detection and the inherent difficulty of capturing detailed object features using robotic sensors. We leverage currently available 3D resources to generate 2D data and train a detector, which is then used to augment the standard 3D scene graph generation pipeline. Through our experiments, we demonstrate that our approach achieves functional element segmentation comparable to state-of-the-art 3D models and that our augmentation enables task-driven affordance grounding with higher accuracy than the current solutions. © 2025, CC BY-NC-ND.

关键词： Semantics

Pedestrians and Robots: A Novel Dataset for learning Distinct Social Navigation Forces

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Agrawal, Subham Ostermann-Myrau, Nico Dengler, Nils Bennewitz, Maren Humanoid Robots Lab University of Bonn Germany Lamarr Institute for Machine Learning and Artificial Intelligence The Center for Robotics Bonn Germany

The increasing use of robots in human-centric public spaces such as shopping malls, sidewalks, and hospitals, requires understanding of how pedestrians respond to their presence. However, existing research lacks comprehensive datasets that capture the full range of pedestrian behaviors, e.g., including avoidance, neutrality, and attraction in the presence of robots. Such datasets can be used to effectively learn models capable of accurately predicting diverse responses of pedestrians to robot presence, which are crucial for advancing robot navigation strategies and optimizing pedestrian-aware motion planning. In this paper, we address these challenges by collecting a novel dataset of pedestrian motion in two outdoor locations under three distinct conditions, i.e., no robot presence, a stationary robot, and a moving robot. Thus, unlike existing datasets, ours explicitly encapsulates variations in pedestrian behavior across the different robot conditions. Using our dataset, we propose a novel Neural Social Robot Force Model (NSRFM), an extension of the traditional Social Force Model that integrates neural networks and robot-induced forces to better predict pedestrian behavior in the presence of robots. We validate the NSRFM by comparing its generated trajectories on different real-world datasets. Furthermore, we implemented it in simulation to enable the learning and benchmarking of robot navigation strategies based on their impact on pedestrian movement. Our results demonstrate the model’s effectiveness in replicating real-world pedestrian reactions and its its utility in developing, evaluating, and benchmarking social robot navigation algorithms. Copyright © 2025, The Authors. All rights reserved.

关键词： Social robots

Cross-Attention Between Satellite and Ground Views for Enhanced Fine-Grained Robot Geo-Localization

学校读者我要写书评

暂无评论

Cross-Attention Between Satellite and Ground Views for Enhan...

IEEE Workshop on Applications of Computer Vision (WACV)

作者： Dong Yuan Frederic Maire Feras Dayoub QUT Centre for Robotics Queensland University of Technology Australia Australian Institute for Machine Learning (AIML) University of Adelaide Australia

Cross-view image geo-localization aims to determine the locations of outdoor robots by mapping current street-view images with GPS-tagged satellite image patches. Recent works have attained a remarkable level of accuracy in identifying which satellite patches the robot is in, where the location of the central pixel within the matched satellite patch is used as the robot coarse location estimation. This work focuses on robot fine-grained localization within a known satellite patch. Existing fine-grain localization work utilizes correlation operation to obtain similarity between satellite image local descriptors and street-view global descriptors. The correlation operation based on liner matching simplifies the interaction process between two views, leading to a large distance error and affecting model generalization. To address this issue, we devise a cross-view feature fusion network with self-attention and cross-attention layers to replace correlation operation. Additionally, we combine classification and regression prediction to further decrease location distance error. Experiments show that our novel network architecture outperforms the state-of-the-art, exhibiting better generalization capabilities in unseen areas. Specifically, our method reduces the median localization distance error by 43% and 50% respectively in the same area and unseen areas on the VIGOR benchmark.

关键词：

A Comparison of Prompt Engineering Techniques for Task Planning and Execution in Service robotics

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Bode, Jonas Pätzold, Bastian Memmesheimer, Raphael Behnke, Sven The Autonomous Intelligent Systems group Computer Science Institute VI – Intelligent Systems and Robotics Lamarr Institute for Machine Learning and Artificial Intelligence Center for Robotics University of Bonn Germany

Recent advances in Large Language Models (LLMs) have been instrumental in autonomous robot control and human-robot interaction by leveraging their vast general knowledge and capabilities to understand and reason across a wide range of tasks and scenarios. Previous works have investigated various prompt engineering techniques for improving the performance of LLMs to accomplish tasks, while others have proposed methods that utilize LLMs to plan and execute tasks based on the available functionalities of a given robot platform. In this work, we consider both lines of research by comparing prompt engineering techniques and combinations thereof within the application of high-level task planning and execution in service robotics. We define a diverse set of tasks and a simple set of functionalities in simulation, and measure task completion accuracy and execution time for several state-of-the-art models. We make our code, including all prompts, available at https://***/AIS-Bonn/Prompt_Engineering. Copyright © 2024, The Authors. All rights reserved.

关键词： Human robot interaction

Temporal Attention for Cross-View Sequential Image Localization

学校读者我要写书评

暂无评论

Temporal Attention for Cross-View Sequential Image Localizat...

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

作者： Dong Yuan Frederic Maire Feras Dayoub QUT Centre for Robotics Queensland University of Technology Australia Australian Institute for Machine Learning (AIML) University of Adelaide Australia

ISBN: (数字)9798350377705

ISBN: (纸本)9798350377712

This paper introduces a novel approach to enhancing cross-view localization, focusing on the fine-grained, sequential localization of street-view images within a single known satellite image patch, a significant departure from traditional one-to-one image retrieval methods. By expanding to sequential image fine-grained localization, our model, equipped with a novel Temporal Attention Module (TAM), leverages contextual information to significantly improve sequential image localization accuracy. Our method shows substantial reductions in both mean and median localization errors on the Cross-View Image Sequence (CVIS) dataset, outperforming current state-of-the-art single-image localization techniques. Additionally, by adapting the KITTI-CVL dataset into sequential image sets, we not only offer a more realistic dataset for future research but also demonstrate our model’s robust generalization capabilities across varying times and areas, evidenced by a 75.3% reduction in mean distance error in cross-view sequential image localization.

关键词： Location awareness Adaptation models Accuracy Image retrieval Focusing Satellite images Image sequences Intelligent robots Context modeling

Security Fence Inspection at Airports Using Object Detection

学校读者我要写书评

暂无评论

Security Fence Inspection at Airports Using Object Detection

IEEE Winter Applications and Computer Vision Workshops (WACVW)

作者： Nils Friederich Andreas Specker Jürgen Beyerer Karlsruhe Institute of Technology Institute for Automation and Applied Informatics Fraunhofer IOSB Fraunhofer Center for Machine Learning Karlsruhe Institute of Technology Institute for Anthropomatics and Robotics

To ensure the security of airports, it is essential to protect the airside from unauthorized access. For this pur-pose, security fences are commonly used, but they require regular inspection to detect damages. However, due to the growing shortage of human specialists and the large man-ual effort, there is the need for automated methods. The aim is to automatically inspect the fence for damage with the help of an autonomous robot. In this work, we explore object detection methods to address the fence inspection task and localize various types of damages. In addition to evaluating four State-of-the-Art (SOTA) object detection models, we analyze the impact of several design criteria, aiming at adapting to the task-specific challenges. This in-cludes contrast adjustment, optimization of hyperparameters, and utilization of modern backbones. The experimental results indicate that our optimized You Only Look Once v5 (YOLOv5) model achieves the highest accuracy of the four methods with an increase of 6.9% points in Average Precision (AP) compared to the baseline. Moreover, we show the real-time capability of the model. The trained models are published on GitHub: hups://***/IN-Friederichlairport_fence_inspection.

关键词：

Fast Global Point Cloud Registration using Semantic NDT

学校读者我要写书评

暂无评论

Fast Global Point Cloud Registration using Semantic NDT

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

作者： Robert Schirmer Narunas Vaskevicius Peter Biber Cyrill Stachniss Robert Bosch GmbH Center for Robotics University of Bonn Germany Lamarr Institute for Machine Learning and Artificial Intelligence Germany

ISBN: (数字)9798350377705

ISBN: (纸本)9798350377712

Robust and accurate point cloud registration is an essential part of many robotic tasks such as SLAM or object pose retrieval. In this paper, we address the problem of global 3D point cloud registration, i.e., the task of estimating the 3D rigid body transform between a source and a target point cloud without any initial guess. Typically, the problem is solved by extracting and matching features to find a data association and then computing a transform that minimizes the squared distance between points. Our approach combines the normal distributions transform and oriented point pair framework and introduces the NDT distance histogram to quickly generate and test candidate transforms. Our method further exploits semantic information if available for greater speed. We implement our algorithm in C++ and compare it to other state-of-the-art approaches on a diverse set of environments. Our evaluation shows that our method outperforms the other approaches, especially concerning run-time and compute efficiency.

关键词： Point cloud compression Location awareness Histograms Three-dimensional displays Simultaneous localization and mapping Semantic segmentation Semantics Transforms Feature extraction Intelligent robots

Self-Centering 3-DoF Feet Controller for Hands-Free Locomotion Control in Telepresence and Virtual Reality

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Memmesheimer, Raphael Lenz, Christian Schwarz, Max Schreiber, Michael Behnke, Sven The Autonomous Intelligent Systems group Computer Science Institute VI - Intelligent Systems and Robotics Lamarr Institute for Machine Learning and Artificial Intelligence Center for Robotics University of Bonn Germany

We present a novel seated feet controller for handling 3 Degree of Freedom (DoF) aimed to control locomotion for telepresence robotics and virtual reality environments. Tilting the feet on two axes yields in forward, backward and sideways motion. In addition, a separate rotary joint allows for rotation around the vertical axis. Attached springs on all joints self-center the controller. The HTC Vive tracker is used to translate the trackers' orientation into locomotion commands. The proposed self-centering feet controller was used successfully for the ANA Avatar XPRIZE competition, where a naive operator traversed the robot through a longer distance, surpassing obstacles while solving various interaction and manipulation tasks in between. We publicly provide the models of the mostly 3D-printed feet controller for reproduction. Copyright © 2024, The Authors. All rights reserved.

关键词： Biped locomotion

Grasp Anything: Combining Teacher-Augmented Policy Gradient learning with Instance Segmentation to Grasp Arbitrary Objects

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Mosbach, Malte Behnke, Sven The Autonomous Intelligent Systems group Computer Science Institute VI - Intelligent Systems and Robotics The Center for Robotics The Lamarr Institute for Machine Learning and Artificial Intelligence University of Bonn Germany

Interactive grasping from clutter, akin to human dexterity, is one of the longest-standing problems in robot learning. Challenges stem from the intricacies of visual perception, the demand for precise motor skills, and the complex interplay between the two. In this work, we present Teacher-Augmented Policy Gradient (TAPG), a novel two-stage learning framework that synergizes reinforcement learning and policy distillation. After training a teacher policy to master the motor control based on object pose information, TAPG facilitates guided, yet adaptive, learning of a sensorimotor policy, based on object segmentation. We zero-shot transfer from simulation to a real robot by using Segment Anything Model for promptable object segmentation. Our trained policies adeptly grasp a wide variety of objects from cluttered scenarios in simulation and the real world based on human-understandable prompts. Furthermore, we show robust zero-shot transfer to novel objects. Videos of our experiments are available at https://***/grasp_anything. Copyright © 2024, The Authors. All rights reserved.

关键词： Distillation