检索结果-内蒙古大学图书馆

MVF-GNN: Multi-View Fusion With GNN for 3D Semantic Segmentation

IEEE ROBOTICS AND AUTOMATION LETTERS 2025年第4期10卷 3262-3269页

作者： Du, Zhenxiang Ren, Minglun Chu, Wei Chen, Nengying Hefei Univ Technol Sch Management Hefei 230009 Peoples R China

Due to the high cost of obtaining 3D annotations and the accumulation of many 2D datasets with 2D semantic labels, deploying multi-view 2D images for 3D semantic segmentation has attracted widespread attention. Fusion of multi-view information requires establishing local-to-local as well as local-to-global dependencies among multiple views. However, previous methods that are based on 2D annotations supervision cannot model local-to-local and local-to-global dependencies simultaneously. In this letter, we propose a novel multi-view fusion framework with graph neural networks (MVF-GNN) for multi-view interaction and integration. First, a multi-view graph based on the associated pixels in multiple views is constructed. Then, a multi-scale multi-view graph attention network (MSMVGAT) module is introduced to perform graph reasoning on multi-view graphs at different scales. Finally, an attention multi-view graph aggregation (AMVGA) module is introduced to learn the importance of different views and integrate multi-view features. Experiments on the ScanNetv2 benchmark dataset show that our method outperforms state-of-the-art 2D/3D semantic segmentation methods based on 2D annotations supervision.

关键词： Three-dimensional displays Semantics Semantic segmentation Graph neural networks Feature extraction Geometry Solid modeling Convolution Cameras Cognition deep learning for visual perception Semantic Scene Understanding

来源：评论

学校读者我要写书评

暂无评论

Matching Distance and Geometric Distribution Aided learning Multiview Point Cloud Registration

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第11期9卷 9319-9326页

作者： Li, Shiqi Zhu, Jihua Xie, Yifan Hu, Naiwen Wang, Di Xi An Jiao Tong Univ Sch Software Engn Xian 710049 Peoples R China Xi An Jiao Tong Univ Shaanxi Joint Key Lab Artifact Intelligence Xian 710049 Peoples R China

Multiview point cloud registration plays a crucial role in robotics, automation, and computer vision fields. This letter concentrates on pose graph construction and motion synchronization within multiview registration. Previous methods for pose graph construction often pruned fully connected graphs or constructed sparse graph using global feature aggregated from local descriptors, which may not consistently yield reliable results. To identify dependable pairs for pose graph construction, we design a network model that extracts information from the matching distance between point cloud pairs. For motion synchronization, we propose another neural network model to calculate the absolute pose in a data-driven manner, rather than optimizing inaccurate handcrafted loss functions. Our model takes into account geometric distribution information and employs a modified attention mechanism to facilitate flexible and reliable feature interaction. Experimental results on diverse indoor and outdoor datasets confirm the effectiveness and generalizability of our approach.

关键词： Point cloud compression Synchronization Reliability Feature extraction Data mining Standards Training deep learning for visual perception multiview registration point cloud registration

来源：评论

学校读者我要写书评

暂无评论

Automated Generation of Transformations to Mitigate Sensor Hardware Migration in ADS

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第7期9卷 6480-6487页

作者： von Stein, Meriel Wang, Hongning Elbaum, Sebastian Univ Virginia Sch Engn Comp Sci Dept Charlottesville VA 22902 USA

Autonomous driving systems (ADSs) rely on massive amounts of sensed data to train their underlying deep neural networks (DNNs). Common sensor hardware migrations can render an existing DNN-dependent pipeline inadequate. This necessitates the development of bespoke transformations to adapt new sensor data to the old trained network, or the retraining of a new network with new sensor data. These solutions are expensive, often performed reactively to sensor hardware migration, and rely only on empirical reconstruction and validation metrics which lack knowledge of the features important to the trained DNN. To address these challenges, we propose PreFixer, a technique that can systematically generate transformations for many types of sensor hardware migration during the ADS development lifecycle. PreFixer collects small datasets using colocated new and old sensors, and then uses that data and the output of the original trained DNN to train an augmented encoder to learn a transformation that maps new sensor data to old sensor data. The trained encoder can then be deployed as a preprocessor to the original trained DNN. Our study shows that, for a common set of camera sensor hardware migrations, PreFixer can match or improve the performance of the best-performing specialized baseline technique in terms of distance travelled safely with 10% of the training dataset, and take at most half of the training time of a new network.

关键词： Autonomous vehicle navigation deep learning for visual perception vision-based navigation

来源：评论

学校读者我要写书评

暂无评论

Using a Distance Sensor to Detect Deviations in a Planar Surface

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第10期9卷 8515-8522页

作者： Sifferman, Carter Sun, William Gupta, Mohit Gleicher, Michael Univ Wisconsin Dept Comp Sci Madison WI 53706 USA

We investigate methods for determining if a planar surface contains geometric deviations (e.g., protrusions, objects, divots, or cliffs) using only an instantaneous measurement from a miniature optical time-of-flight sensor. The key to our method is to utilize the entirety of information encoded in raw time-of-flight data captured by off-the-shelf distance sensors. We provide an analysis of the problem in which we identify the key ambiguity between geometry and surface photometrics. To overcome this challenging ambiguity, we fit a Gaussian mixture model to a small dataset of planar surface measurements. This model implicitly captures the expected geometry and distribution of photometrics of the planar surface and is used to identify measurements that are likely to contain deviations. We characterize our method on a variety of surfaces and planar deviations across a range of scenarios. We find that our method utilizing raw time-of-flight data outperforms baselines which use only derived distance estimates. We build an example application in which our method enables mobile robot obstacle and cliff avoidance over a wide field-of-view.

关键词： Range sensing deep learning for visual perception vision-based navigation direct time-of-flight Range sensing deep learning for visual perception vision-based navigation direct time-of-flight

来源：评论

学校读者我要写书评

暂无评论

Interdimensional Knowledge Transfer for Semantic Segmentation on LiDAR Point Clouds

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第9期9卷 7501-7508页

作者： Ha, Seongheon Kim, Yeogyeong Park, Jinsun Pusan Natl Univ Dept Informat Convergence Engn Artificial Intelligence Major Busan 46241 South Korea Pusan Natl Univ Sch Comp Sci & Engn Busan 46241 South Korea Pusan Natl Univ Ctr Artificial Intelligence Res Busan 46241 South Korea

3D semantic segmentation is a task to classify a set of 3D points in a scene into semantically meaningful objects. Most existing approaches solely rely on single-domain information such as point clouds acquired by LiDAR sensors. However, the amount of information acquired from a single domain information is inherently limited. The limitation can be alleviated by complementary knowledge exchange between multi-modal sensors such as RGB and LiDAR sensors, but this leads to increased computational complexity inevitably. To address the problem, we propose a novel Interdimensional Alignment Module (IAM) for knowledge transfer between 2D and 3D information. The proposed IAM aims to exploit multi-modal information during the training by combining complementary 2D and 3D information based on knowledge distillation with a teacher-student structure while utilizing single-domain information during the inference stage to keep the computational cost low. Specifically, the teacher network utilizes 3D point clouds projected onto the paired 2D image to extract geometrically aligned key-value pairs from each modality. After that, the extracted global context vectors are exchanged between modalities to generate cross-modal attention-based 2D-3D fused features. The prediction from the fused features is adopted as unidirectional guidance to the lightweight student network, therefore, only 3D information is required for the inference. We also design a FocalKLD loss to effectively deal with the class imbalance problem during the distillation. This leads to significantly improved performance compared to previous single-domain segmentation methods. Experimental results on SemanticKITTI and nuScenes datasets show that our model is effective and superior to previous methods.

关键词： Three-dimensional displays Point cloud compression Laser radar Sensors Feature extraction Knowledge transfer Semantic segmentation deep learning for visual perception semantic scene understanding AI-based methods

来源：评论

学校读者我要写书评

暂无评论

Cherry CO Dataset: A Dataset for Cherry Detection, Segmentation and Maturity Recognition

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第6期9卷 5552-5558页

作者： Cossio-Montefinale, Luis Ruiz-del-Solar, Javier Verschae, Rodrigo Univ Chile Dept Elect Engn Santiago 8330015 Chile Univ Chile Adv Min Technol Ctr AMTC Santiago 8330015 Chile Univ OHiggins Inst Engn Sci Rancagua 2820000 Chile

In recent years, there has been an increasing interest in using robotics and autonomous systems for fruit farming. These systems use detection and segmentation algorithms to provide computers with a way to interpret and interact with plants, thus allowing them to automatize assessment and harvesting tasks. This type of algorithm requires a dataset to train a machine learning algorithm, but most types of fruits lack a publicly available dataset. We present Cherry CO, a novel, high-resolution dataset for cherry detection and segmentation. This dataset contains 3,006 labeled images that were acquired in a cherry plantation. The images were taken in various conditions to account for most challenges in this environment. Each cherry was manually labeled to account for location, shape and classes, taking into account aspects such as ripeness, health state and location. This letter presents a detailed description of data acquisition, annotation specifications and statistical analysis of the dataset. In addition, we trained and evaluated several deep Neural Networks to provide a benchmark for cherry detection, achieving high performance with most networks, most notably YOLOv7. The dataset and evaluation software are provided for research purposes.

关键词： Agricultural automation data sets for robot learning deep learning for visual perception object detection segmentation and categorization

来源：评论

学校读者我要写书评

暂无评论

VRVP: Valuable Region and Valuable Point Anchor-Free 3D Object Detection

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第1期9卷 33-40页

作者： Deng, Pengzhen Zhou, Li Chen, Jie Chinese Acad Sci Inst Microelect Beijing 100029 Peoples R China Univ Chinese Acad Sci Beijing 100049 Peoples R China

3D point cloud object detection is of great significance in autonomous driving, robotics, and related fields. However, the current algorithms fail to fully consider that positive sample points of a point cloud object exclusively reside on its surface, resulting in decreased accuracy. High-quality classification points are located centrally, while high-quality regression points are predominantly situated at the boundary. We propose an anchor-free object detection algorithm called Valuable Region and Valuable Point (VRVP), aiming to address the inconsistency between the classification and regression tasks. We achieve high-quality object size regression through a classification-based approach and implicitly correlate the regression and classification branches. Furthermore, a valuable region extraction module is introduced to select valuable points and regions and fill in the missing features of the object's center area. We validate the effectiveness of our algorithm on both the KITTI dataset and Waymo dataset by comparing it with state-of-the-art algorithms. The results demonstrate that the proposed method exhibits strong competitiveness by significantly improving detection accuracy for small objects such as pedestrians and cyclists, while maintaining high accuracy in detecting cars.

关键词： Feature extraction Three-dimensional displays Point cloud compression Object detection Task analysis Convolution Prediction algorithms Computer vision for automation deep learning for visual perception object detection segmentation and categorization

来源：评论

学校读者我要写书评

暂无评论

GISR: Geometric Initialization and Silhouette-Based Refinement for Single-View Robot Pose and Configuration Estimation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第11期9卷 9883-9890页

作者： Bilic, Ivan Maric, Filip Bonsignorio, Fabio Petrovic, Ivan Univ Zagreb Fac Elect Engn & Comp Lab Autonomous Syst & Mobile Robot Zagreb 10000 Croatia Univ Toronto Inst Aerosp Studies Space & Terr Autonomous Robot Syst Lab Toronto ON Canada

In autonomous robotics, measurement of the robot's internal state and perception of its environment, including interaction with other agents such as collaborative robots, are essential. Estimating the pose of the robot arm from a single view has the potential to replace classical eye-to-hand calibration approaches and is particularly attractive for online estimation and dynamic environments. In addition to its pose, recovering the robot configuration provides a complete spatial understanding of the observed robot that can be used to anticipate the actions of other agents in advanced robotics use cases. Furthermore, this additional redundancy enables the planning and execution of recovery protocols in case of sensor failures or external disturbances. We introduce GISR - a deep configuration and robot-to-camera pose estimation method that prioritizes execution in real-time. GISR consists of two modules: (i) a geometric initialization module that efficiently computes an approximate robot pose and configuration, and (ii) a deep iterative silhouette-based refinement module that arrives at a final solution in just a few iterations. We evaluate GISR on publicly available data and show that it outperforms existing methods of the same class in terms of both speed and accuracy, and can compete with approaches that rely on ground-truth proprioception and recover only the pose.

关键词： Robots Robot kinematics Rendering (computer graphics) Pose estimation Kinematics Cameras Three-dimensional displays AI-enabled robotics deep learning for visual perception visual learning

来源：评论

学校读者我要写书评

暂无评论

RTONet: Real-Time Occupancy Network for Semantic Scene Completion

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第10期9卷 8370-8377页

作者： Lai, Quan Zheng, Haifeng Feng, Xinxin Zheng, Mingkui Chen, Huacong Chen, Wenqiang Fuzhou Univ Coll Phys & Informat Engn Fujian Key Lab Intelligent Proc & Wireless Transmi Fuzhou 350108 Peoples R China Fujian Hantewin Intelligent Technol Co Ltd Fuzhou 350001 Peoples R China

The comprehension of 3D semantic scenes holds paramount significance in autonomous driving and robotics technology. Nevertheless, the simultaneous achievement of real-time processing and high precision in complex, expansive outdoor environments poses a formidable challenge. In response to this challenge, we propose a novel occupancy network named RTONet, which is built on a teacher-student model. To enhance the ability of the network to recognize various objects, the decoder incorporates dilated convolution layers with different receptive fields and utilizes a multi-path structure. Furthermore, we develop an automatic frame selection algorithm to augment the guidance capability of the teacher network. The proposed method outperforms the existing grid-based approaches in semantic completion (mIoU), and achieves the state-of-the-art performance in terms of real-time inference speed while exhibiting competitive performance in scene completion (IoU) on the SemanticKITTI benchmark.

关键词： Three-dimensional displays Semantics Laser radar Decoding Point cloud compression Feature extraction Real-time systems Semantic scene understanding mapping occupancy grid deep learning for visual perception LiDAR

来源：评论

学校读者我要写书评

暂无评论

A Sparse Octree-Based CNN for Probabilistic Occupancy Prediction Applied to Next Best View Planning

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第11期9卷 9359-9366页

作者： Monica, Riccardo Aleotti, Jacopo Univ Parma Dept Engn & Architecture RIMLab I-43124 Parma Italy

This work proposes OcLe-CNN, a sparse octree-based Convolutional Neural Network (CNN) for 3D occupancy prediction. Occupancy prediction involves the inference of the occupancy probability of unobserved space. OcLe-CNN processes an octree-like data structure resulting in a reduced memory usage, as resources are allocated prevalently in the most detail-rich regions of the environment. Also, a novel loss function is introduced which results in smaller octrees compared to the state-of-the-art Structure and Task loss. The proposed CNN was integrated with a probabilistic robot Next Best View (NBV) planner, where an octree-like data structure speeds up the ray casting stage. The integration resulted in a lower total computation time. The method was implemented for both quadtrees and octrees, and it was validated on 2D and 3D datasets as well as on a real robot manipulator setup.

关键词： Octrees Three-dimensional displays Planning Convolution Probabilistic logic Convolutional neural networks Standards deep learning for visual perception view planning for SLAM RGB-D perception

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：