检索结果-内蒙古大学图书馆

Robot Grasp in Cluttered Scene Using a Multi-Stage deep learning Model

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第7期9卷 6512-6519页

作者： Wei, Dujia Cao, Jianmin Gu, Ye Shenzhen Technol Univ Coll Big Data & Internet Shenzhen 518118 Peoples R China

Object grasping in cluttered scene is a practical robotic skill which has a wide range of applications. In this paper, we propose a novel maximum graspness metric which can help extract high-quality scene grasp points effectively. The graspness scores of a single-view point cloud are generated using the proposed interpolation approach. The graspness model is implemented using a compact encoder-decoder model which takes a depth image as input. On the other hand, the grasp point features are extracted. They are further grouped and sampled to predict approaching vectors and in-plane rotations of the grasp poses using residual point blocks. The proposed model is evaluated using a large scale benchmark GraspNet-1Billion dataset and can outperform prior state-of-the-art method by a margin (+4.91 AP) on all camera types. Through real-world cluttered scenario testing, our approach achieves grasping successful rate of 89.60% using a UR-5 robotic arm and a RealSense camera.

关键词： deep learning for visual perception computer vision for automation perception for grasping manipulation

来源：评论

学校读者我要写书评

暂无评论

Panoptic Vision-Language Feature Fields

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第3期9卷 2144-2151页

作者： Chen, Haoran Blomqvist, Kenneth Milano, Francesco Siegwart, Roland Swiss Fed Inst Technol Autonomous Syst Lab CH-8092 Zurich Switzerland

Recently, methods have been proposed for 3D open-vocabulary semantic segmentation. Such methods are able to segment scenes into arbitrary classes based on text descriptions provided during runtime. In this letter, we propose to the best of our knowledge the first algorithm for open-vocabulary panoptic segmentation in 3D scenes. Our algorithm, Panoptic Vision-Language Feature Fields (PVLFF), learns a semantic feature field of the scene by distilling vision-language features from a pretrained 2D model, and jointly fits an instance feature field through contrastive learning using 2D instance segments on input frames. Despite not being trained on the target classes, our method achieves panoptic segmentation performance similar to the state-of-the-art closed-set 3D systems on the HyperSim, ScanNet and Replica dataset and additionally outperforms current 3D open-vocabulary systems in terms of semantic segmentation. We ablate the components of our method to demonstrate the effectiveness of our model architecture.

关键词： Semantics Three-dimensional displays Semantic segmentation Self-supervised learning Instance segmentation Image reconstruction Computational modeling Semantic scene understanding deep learning for visual perception 3D open vocabulary panoptic segmentation neural implicit representation

来源：评论

学校读者我要写书评

暂无评论

Toward Robust Keypoint Detection and Tracking: A Fusion Approach With Event-Aligned Image Features

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第9期9卷 8059-8066页

作者： Wang, Xiangyuan Yu, Huai Yu, Lei Yang, Wen Xia, Gui-Song Wuhan Univ Sch Elect Informat Wuhan 430072 Peoples R China Wuhan Univ Sch Comp Sci Wuhan 430072 Peoples R China

Robust keypoint detection and tracking are crucial for various robotic tasks. However, conventional cameras struggle under rapid motion and lighting changes, hindering local and edge feature extraction essential for keypoint detection and tracking. Event cameras offer advantages in such scenarios due to their high dynamic range and low latency. Yet, their inherent noise and motion dependence can lead to feature instability. This letter presents a novel image-event fusion approach for robust keypoint detection and tracking under challenging conditions. We leverage the complementary strengths of image and event data by introducing: (i) the Implicit Compensation Module and Temporal Alignment Module for high-frequency feature fusion and keypoint detection;and (ii) a temporal neighborhood matching strategy for robust keypoint tracking within a sliding window. Furthermore, a self-supervised temporal response consistency constraint ensures keypoint continuity and stability. Extensive experiments demonstrate the effectiveness of our method against state-of-the-art approaches under diverse challenging scenarios. In particular, our method exhibits the longest tracking lifetime and a strong generalizability on real-world data.

关键词： Tracking Feature extraction Cameras Image resolution Location awareness Lighting Target tracking deep learning for visual perception keypoint detection and tracking image-event fusion event camera

来源：评论

学校读者我要写书评

暂无评论

Camera-Based Online Vectorized HD Map Construction With Incomplete Observation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第7期9卷 6456-6463页

作者： Liu, Hui Chang, Faliang Liu, Chunsheng Lu, Yansha Liu, Minhang Shandong Univ Sch Control Sci & Engn Jinan 250061 Peoples R China

Camera-based online map construction focuses on learning map elements from surround-view images. Distinguished with previous methods that rely on complete observations, we explore a new map construction problem under incomplete observations where one or more perspectives of the surround-view are missing due to camera damage or occlusion. Incomplete observations lead to inferior performance and may even result in failure. Map construction based on incomplete observations faces two challenges: supplementing missing perspective features and reducing the complexity of high-dimensional feature learning. To address these issues, we propose a novel Panoramic Observation Prior Network (POP-Net). Firstly, based on the observation switch training mechanism, we propose a Panoramic learning Module (PL-Module). It establishes a learnable panoramic feature space, facilitating the extraction of panoramic features from incomplete observations, thus supplementing missing perspective features. Secondly, based on the feature decomposition mechanism, we design a Panoramic Decomposition-Aggregation Operation (PDA-Operation), which decomposes high-dimensional panoramic features into low-dimensional local scene features. This allows limited local scene features to represent diverse panoramic features, alleviating computational and memory burdens of high-dimensional feature learning. Experimental results demonstrate that our method surpasses existing approaches under incomplete observation scenarios.

关键词： Feature extraction Cameras Training Laser radar Decoding Switches Representation learning Intelligent transportation systems semantic scene understanding deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

Panoptic Out-of-Distribution Segmentation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第5期9卷 4075-4082页

作者： Mohan, Rohit Kumaraswamy, Kiran Hurtado, Juana Valeria Petek, Kursat Valada, Abhinav Univ Freiburg Dept Comp Sci D-79110 Freiburg Germany

deep learning has led to remarkable strides in scene understanding with panoptic segmentation emerging as a key holistic scene interpretation task. However, the performance of panoptic segmentation is severely impacted in the presence of out-of-distribution (OOD) objects i.e. categories of objects that deviate from the training distribution. To overcome this limitation, we propose panoptic out-of-distribution segmentation for joint pixel-level semantic in-distribution and out-of-distribution classification with instance prediction. We extend two established panoptic segmentation benchmarks, Cityscapes and BDD100 K, with out-of-distribution instance segmentation annotations, propose suitable evaluation metrics, and present multiple strong baselines. Importantly, we propose the novel PoDS architecture with a shared backbone, an OOD contextual module for learning global and local OOD object cues, and dual symmetrical decoders with task-specific heads that employ our alignment-mismatch strategy for better OOD generalization. Combined with our data augmentation strategy, this approach facilitates progressive learning of out-of-distribution objects while maintaining in-distribution performance. We perform extensive evaluations that demonstrate that our proposed PoDS network effectively addresses the main challenges and substantially outperforms the baselines.

关键词： deep learning for visual perception computer vision for transportation data sets for robotic vision

来源：评论

学校读者我要写书评

暂无评论

learning the Ego-Motion of an Underwater Imaging Sonar: A Comparative Experimental Evaluation of Novel CNN and RCNN Approaches

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第3期9卷 2072-2079页

作者： Munoz, Bastian Troni, Giancarlo Pontificia Univ Catolica Chile Dept Mech & Met Engn Santiago 8320165 Chile Monterey Bay Aquarium Res Inst Moss Landing CA 95039 USA

This research addresses the challenge of estimating the ego-motion of a forward-looking sonar (FLS) through deep neural networks (DNNs) and their application in autonomous underwater robots. Over the last two decades, analytical methods have been developed to perform odometry estimation using FLS data. While these methods can be effective, they are often computationally intensive, complex to implement, or rely on simplifying assumptions restricting their widespread application. Inspired by works on the optical domain, we propose two novel deep-learning approaches to estimate the FLS ego-motion. The first approach employs a convolutional neural network (CNN) to estimate motion from FLS images directly. The second approach leverages sequential image information using a recurrent convolutional neural network (RCNN). We quantitatively evaluate their performance by training and testing on synthetic and field data. Results show that both methods can learn to estimate ego-motion on both data types and that including sequential information can improve performance. This letter presents the first usage of a RCNN for this task, advances toward real-world applications by fine-tuning the models based on field data, and the first quantitative evaluation of these kinds of methods for acoustic odometry using field data.

关键词： Marine robotics deep learning for visual perception localization imaging sonar odometry underwater navigation

来源：评论

学校读者我要写书评

暂无评论

Generalized Synchronized Active learning for Multi-Agent-Based Data Selection on Mobile Robotic Systems

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第10期9卷 8659-8666页

作者： Schmidt, Sebastian Stappen, Lukas Schwinn, Leo Gunnemann, Stephan Tech Univ Munich TUM Sch Computat Informat & Technol D-80333 Munich Germany BMW Grp D-80809 Munich Germany

In mobile robotics, perception in uncontrolled environments like autonomous driving is a central hurdle. Existing active learning frameworks can help enhance perception by efficiently selecting data samples for labeling, but they are often constrained by the necessity of full data availability in data centers, hindering real-time, on-field adaptations. To address this, our work unveils a novel active learning formulation optimized for multi-robot settings. It harnesses the collaborative power of several robotic agents, considerably enhancing the data acquisition and synchronization processes. Experimental evidence indicates that our approach markedly surpasses traditional active learning frameworks by up to 2.5 percent points and 90% less data uploads, delivering new possibilities for advancements in the realms of mobile robotics and autonomous systems.

关键词： Computer vision for transportation deep learning for visual perception deep learning methods Computer vision for transportation deep learning for visual perception deep learning methods

来源：评论

学校读者我要写书评

暂无评论

GeoCluster: Enhancing visual Place Recognition in Spatial Domain on Aerial Vehicle Platforms

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第3期9卷 3013-3020页

作者： Chen, Chao He, Mengfan Wang, Jun Meng, Ziyang Beijing Univ Chem Technol Coll Informat Sci &Technol Beijing 100029 Peoples R China Tsinghua Univ Dept Precis Instrument Beijing 100084 Peoples R China

visual Place Recognition (VPR) is a critical technology for achieving robust long-term visual geo-localization. During the past few years, VPR research mainly focused on ground-based platforms in the street-level captured scenes with deep learning methods (e.g. NetVLAD, GeM), but little attention was paid to the VPR task on aerial vehicles. The algorithms and models designed for ground-based platforms are always directly applied to the aerial VPR problem. However, the viewpoint variance on Unmanned Aerial Vehicles (UAV) is much larger than the ground-based platforms. Due to the sparse distribution of aerial image features, when the viewpoint of the camera changes, the features of the query image are largely inconsistent with the descriptors in the database, which results in the failures of image retrieval and visual geo-localization. In this letter, we propose an aerial VPR enhancement module called GeoCluster, which presents a feature aggregation method using spatial clustering information to improve the robustness and consistency of the global descriptors for UAV-captured frames. Moreover, it can be applied to any NetVLAD-based VPR method and boost the pre-trained model without any further training process. By integrating GeoCluster into an existing state-of-the-art localization method, we can achieve about 10% improvement for aerial image retrieval tasks and have more accurate and robust geo-localization results.

关键词： Vision-based navigation localization recognition deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

Leveraging Map Retrieval and Alignment for Robust UAV visual Geo-Localization

引用

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT 2024年 73卷 1页

作者： He, Mengfan Liu, Jiacheng Gu, Pengfei Meng, Ziyang Tsinghua Univ Dept Precis Instrument Beijing 100084 Peoples R China Unmanned Aerial Vehicle Lab Beijing 100012 Peoples R China

The autonomous flight of unmanned aerial vehicles (UAVs) relies on a precise and robust geo-localization system. A visual geo-localization system registers the aerial captured image to a geo-referenced satellite map, which enables UAVs to determine their global position without a global navigation satellite system (GNSS). However, it is challenging to achieve precise and robust geo-localization in a large-scale environment due to the notable texture difference between the satellite map and the UAV-captured image. In this article, we design a robust visual geo-localization pipeline that integrates a proposed deep learning-based imagery feature. This pipeline starts with the image retrieval based on the deep feature encoding, to initialize the localization process over large-scale maps without any location prior. With the reuse of the same deep imagery feature, an image registration process enables real-time sequential localization. Besides, the proposed system has the re-localization ability to eliminate the localization drift caused by possible registration failure, especially during long-time flights. Evaluations of datasets and real-world experiments demonstrate that the proposed system is more robust and accurate than other state-of-the-art methods. To encourage further progress on the visual geo-localization problem, our code and materials are publicly available at https://***/hmf21/UAVLocalization.

关键词： Location awareness visualization Autonomous aerial vehicles Feature extraction Accuracy Image retrieval Satellites deep learning for visual perception hierarchical localization map-based navigation shared encoder visual geo-localization

来源：评论

学校读者我要写书评

暂无评论

Unbounded-GS: Extending 3D Gaussian Splatting With Hybrid Representation for Unbounded Large-Scale Scene Reconstruction

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第12期9卷 11529-11536页

作者： Li, Wanzhang Yin, Fukun Liu, Wen Yang, Yiying Chen, Xin Jiang, Biao Yu, Gang Fan, Jiayuan Fudan Univ Sch Informat Sci & Technol Shanghai 200433 Peoples R China Platform & Content Grp PCG Tencent Shenzhen 200030 Peoples R China Fudan Univ Acad Engn & Technol Shanghai 200433 Peoples R China

Modeling large-scale scenes from multi-view images is challenging due to the trade-off dilemma between visual quality and computational cost. Existing NeRF-based methods have made advancements in neural implicit representation through volumetric ray-marching, but still struggle to deal with cubically growing sampling space in large-scale scenes. Fortunately, the rendering approach based on 3D Gaussian splatting (3DGS) has shown promising results, inspiring further exploration in the splatting setting. However, 3DGS has the limitation of inadequate Gaussian points for modeling distant backgrounds, leading to "splotchy" artifacts. To address this problem, we introduce a novel hybrid neural representation called Unbounded 3D Gaussian. For foreground area, we employs an explicit 3D Gaussian representation to efficiently model the geometry and appearance through splatting weighted Gaussians. For far-away background, we additionally introduce an implicit module comprising Multi-layer perceptions (MLPs) to directly predict far-away background colors from positional encodings of view positions and ray directions. Furthermore, we design a seamless blending mechanism between the color predictions of the explicit splatting and implicit branches to reconstruct holistic scenes. Extensive experiments demonstrate that our proposed Unbounded-GS inherits the advantages of both faster convergence and high-fidelity rendering quality.

关键词： deep learning for visual perception visual learning visual learning view synthesis view synthesis 3D reconstruction 3D reconstruction 3D reconstruction

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：