检索结果-内蒙古大学图书馆

SLCF-Net: Sequential LiDAR-Camera Fusion for Semantic Scene Completion using a 3D Recurrent U-Net

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Cao, Helin Behnke, Sven Autonomous Intelligent Systems group Computer Science Institute VI-Intelligent Systems and Robotics Center for Robotics and the Lamarr Institute for Machine Learning and Artificial Intelligence University of Bonn Germany

We introduce SLCF-Net, a novel approach for the Semantic Scene Completion (SSC) task that sequentially fuses LiDAR and camera data. It jointly estimates missing geometry and semantics in a scene from sequences of RGB images and sparse LiDAR measurements. The images are semantically segmented by a pre-trained 2D U-Net and a dense depth prior is estimated from a depth-conditioned pipeline fueled by Depth Anything. To associate the 2D image features with the 3D scene volume, we introduce Gaussian-decay Depth-prior Projection (GDP). This module projects the 2D features into the 3D volume along the line of sight with a Gaussian-decay function, centered around the depth prior. Volumetric semantics is computed by a 3D U-Net. We propagate the hidden 3D U-Net state using the sensor motion and design a novel loss to ensure temporal consistency. We evaluate our approach on the SemanticKITTI dataset and compare it with leading SSC approaches. The SLCF-Net excels in all SSC metrics and shows great temporal consistency. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics

MOTPose: Multi-object 6D Pose Estimation for Dynamic Video Sequences using Attention-based Temporal Fusion

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Periyasamy, Arul Selvam Behnke, Sven the Autonomous Intelligent Systems group Computer Science Institute VI – Intelligent Systems and Robotics the Center for Robotics the Lamarr Institute for Machine Learning and Artificial Intelligence University of Bonn Germany

Cluttered bin-picking environments are challenging for pose estimation models. Despite the impressive progress enabled by deep learning, single-view RGB pose estimation models perform poorly in cluttered dynamic environments. Imbuing the rich temporal information contained in the video of scenes has the potential to enhance models’ ability to deal with the adverse effects of occlusion and the dynamic nature of the environments. Moreover, joint object detection and pose estimation models are better suited to leverage the co-dependent nature of the tasks for improving the accuracy of both tasks. To this end, we propose attention-based temporal fusion for multi-object 6D pose estimation that accumulates information across multiple frames of a video sequence. Our MOTPose method takes a sequence of images as input and performs joint object detection and pose estimation for all objects in one forward pass. It learns to aggregate both object embeddings and object parameters over multiple time steps using cross-attention-based fusion modules. We evaluate our method on the physically-realistic cluttered bin-picking dataset SynPick and the YCB-Video dataset and demonstrate improved pose estimation accuracy as well as better object detection accuracy. © 2024, CC BY.

关键词： Object detection

DiffSSC: Semantic LiDAR Scan Completion using Denoising Diffusion Probabilistic Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Cao, Helin Behnke, Sven The Autonomous Intelligent Systems group Computer Science Institute VI – Intelligent Systems and Robotics The Center for Robotics The Lamarr Institute for Machine Learning and Artificial Intelligence University of Bonn Germany

Perception systems play a crucial role in autonomous driving, incorporating multiple sensors and corresponding computer vision algorithms. 3D LiDAR sensors are widely used to capture sparse point clouds of the vehicle’s surroundings. However, such systems struggle to perceive occluded areas and gaps in the scene due to the sparsity of these point clouds and their lack of semantics. To address these challenges, Semantic Scene Completion (SSC) jointly predicts unobserved geometry and semantics in the scene given raw LiDAR measurements, aiming for a more complete scene representation. Building on promising results of diffusion models in image generation and super-resolution tasks, we propose their extension to SSC by implementing the noising and denoising diffusion processes in the point and semantic spaces individually. To control the generation, we employ semantic LiDAR point clouds as conditional input and design local and global regularization losses to stabilize the denoising process. We evaluate our approach on autonomous driving datasets and our approach outperforms the state-of-the-art for SSC. © 2024, CC BY.

关键词： Semantics

SLCF-Net: Sequential LiDAR-Camera Fusion for Semantic Scene Completion using a 3D Recurrent U-Net

学校读者我要写书评

暂无评论

SLCF-Net: Sequential LiDAR-Camera Fusion for Semantic Scene ...

IEEE International Conference on robotics and Automation (ICRA)

作者： Helin Cao Sven Behnke Autonomous Intelligent Systems Group Computer Science Institute VI – Intelligent Systems and Robotics – and the Center for Robotics and the Lamarr Institute for Machine Learning and Artificial Intelligence University of Bonn Germany

ISBN: (数字)9798350384574

ISBN: (纸本)9798350384581

关键词： Measurement Image segmentation Three-dimensional displays Laser radar Semantics Pipelines Robot sensing systems

MOTPose: Multi-object 6D Pose Estimation for Dynamic Video Sequences using Attention-based Temporal Fusion

学校读者我要写书评

暂无评论

MOTPose: Multi-object 6D Pose Estimation for Dynamic Video S...

IEEE International Conference on robotics and Automation (ICRA)

作者： Arul Selvam Periyasamy Sven Behnke Autonomous Intelligent Systems Group Computer Science Institute VI – Intelligent Systems and Robotics – and the Center for Robotics and the Lamarr Institute for Machine Learning and Artificial Intelligence University of Bonn Germany

ISBN: (数字)9798350384574

ISBN: (纸本)9798350384581

关键词： Deep learning Accuracy Fuses Aggregates Pose estimation Video sequences Object detection

Grasp Anything: Combining Teacher-Augmented Policy Gradient learning with Instance Segmentation to Grasp Arbitrary Objects

学校读者我要写书评

暂无评论

Grasp Anything: Combining Teacher-Augmented Policy Gradient ...

IEEE International Conference on robotics and Automation (ICRA)

作者： Malte Mosbach Sven Behnke Autonomous Intelligent Systems Group Computer Science Institute VI – Intelligent Systems and Robotics – and the Center for Robotics and the Lamarr Institute for Machine Learning and Artificial Intelligence University of Bonn Germany

ISBN: (数字)9798350384574

ISBN: (纸本)9798350384581

Interactive grasping from clutter, akin to human dexterity, is one of the longest-standing problems in robot learning. Challenges stem from the intricacies of visual perception, the demand for precise motor skills, and the complex interplay between the two. In this work, we present Teacher-Augmented Policy Gradient (TAPG), a novel two-stage learning framework that synergizes reinforcement learning and policy distillation. After training a teacher policy to master the motor control based on object pose information, TAPG facilitates guided, yet adaptive, learning of a sensorimotor policy, based on object segmentation. We zero-shot transfer from simulation to a real robot by using Segment Anything Model for promptable object segmentation. Our trained policies adeptly grasp a wide variety of objects from cluttered scenarios in simulation and the real world based on human-understandable prompts. Furthermore, we show robust zero-shot transfer to novel objects. Videos of our experiments are available at https://***/grasp_anything.

关键词： Training Instance segmentation Motor drives Object segmentation Reinforcement learning Robot sensing systems Motors

Epipolar Attention Field Transformers for Bird’s Eye View Semantic Segmentation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Witte, Christian Behley, Jens Stachniss, Cyrill Raaijmakers, Marvin CARIAD SE Germany Center for Robotics University of Bonn Germany Lamarr Institute for Machine Learning and Artificial Intelligence Germany

Spatial understanding of the semantics of the surroundings is a key capability needed by autonomous cars to enable safe driving decisions. Recently, purely vision-based solutions have gained increasing research interest. In particular, approaches extracting a bird’s eye view (BEV) from multiple cameras have demonstrated great performance for spatial understanding. This paper addresses the dependency on learned positional encodings to correlate image and BEV feature map elements for transformer-based methods. We propose leveraging epipolar geometric constraints to model the relationship between cameras and the BEV by Epipolar Attention Fields. They are incorporated into the attention mechanism as a novel attribution term, serving as an alternative to learned positional encodings. Experiments show that our method EAFormer outperforms previous BEV approaches by 2% mIoU for map semantic segmentation and exhibits superior generalization capabilities compared to implicitly learning the camera configuration. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantic Segmentation

SAFE: Sensitivity-Aware Features for Out-of-Distribution Object Detection

学校读者我要写书评

暂无评论

SAFE: Sensitivity-Aware Features for Out-of-Distribution Obj...

International Conference on Computer Vision (ICCV)

作者： Samuel Wilson Tobias Fischer Feras Dayoub Dimity Miller Niko Sünderhauf QUT Centre for Robotics Queensland University of Technology Australian Institute for Machine Learning University of Adelaide

We address the problem of out-of-distribution (OOD) detection for the task of object detection. We show that residual convolutional layers with batch normalisation produce Sensitivity-Aware FEatures (SAFE) that are consistently powerful for distinguishing in-distribution from out-of-distribution detections. We extract SAFE vectors for every detected object, and train a multilayer perceptron on the surrogate task of distinguishing adversarially perturbed from clean in-distribution examples. This circumvents the need for realistic OOD training data, computationally expensive generative models, or retraining of the base object detector. SAFE outperforms the state-of-the-art OOD object detectors on multiple benchmarks by large margins, e.g. reducing the FPR95 by an absolute 30.6% from 48.3% to 17.7% on the OpenImages dataset.

关键词：

Demonstration-Enhanced Adaptive Multi-Objective Robot Navigation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： de Heuvel, Jorge Sethuraman, Tharun Bennewitz, Maren Humanoid Robots Lab University of Bonn Germany Lamarr Institute for Machine Learning and Artificial Intelligence Center for Robotics Bonn Germany

Preference-aligned robot navigation in human environments is typically achieved through learning-based approaches, utilizing user feedback or demonstrations for personalization. However, personal preferences are subject to change and might even be context-dependent. Yet traditional reinforcement learning (RL) approaches with static reward functions often fall short in adapting to varying user preferences, inevitably reflecting demonstrations once training is completed. This paper introduces a structured framework that combines demonstration-based learning with multi-objective reinforcement learning (MORL). To ensure real-world applicability, our approach allows for dynamic adaptation of the robot navigation policy to changing user preferences without retraining. It fluently modulates the amount of demonstration data reflection and other preference-related objectives. Through rigorous evaluations, including a baseline comparison and simto-real transfer on two robots, we demonstrate our framework’s capability to adapt to user preferences accurately while achieving high navigational performance in terms of collision avoidance and goal pursuance. © 2024, CC BY.

关键词： Reinforcement learning