检索结果-内蒙古大学图书馆

Fine-Grained Semantic Information Preservation and Misclassification-Aware Loss for 3D Point Cloud

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第9期9卷 7947-7954页

作者： Zou, Yanmei Lin, Xuefei Yu, Hongshan Yang, Zhengeng Akhtar, Naveed Hunan Univ Coll Elect & Informat Engn Sch Robot Changsha 410082 Peoples R China Hunan Univ Quanzhou Inst Ind Design & Machine Intelligence In Sch Design Changsha 410082 Peoples R China Hunan Normal Univ Coll Engn & Design Changsha 410081 Peoples R China Univ Melbourne Sch Comp & Informat Syst Melbourne Vic 3052 Australia

Encoder-Decoder structure is a popular choice in point cloud processing for dense multi-classification tasks, e.g., 3D semantic segmentation. Though existing techniques that follow this structure achieve high performance, they are known to suffer from fine-grained information loss, especially when the underlying networks are deep. To alleviate this, we propose a bilateral attention fusion module (BAFM) that leverages multi-resolution feature fusion to allow more effective fine-grained information flow in the network. Moreover, we also introduce a misclassification-aware loss (MAL) as a more potent alternative to the widely used cross-entropy (CE) loss for multi-classification tasks. MAL enables an explicit penalization of misclassification. Empirical experiments reveal that our method achieves state-of-the-art performance for several challenging datasets, such as 91.9% mAcc on ModelNet40, 88.4% OA and 87.0% mAcc on ScanObjectNN and 71.3% mIoU on S3DIS Area-5.

关键词： Point cloud compression Feature extraction Task analysis deep learning for visual perception recognition visual learning

来源：评论

学校读者我要写书评

暂无评论

Focusing on Object Extremities for Tree Instance Segmentation in Forest Environments

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第6期9卷 5480-5487页

作者： Condat, Robin Vasseur, Pascal Allibert, Guillaume Univ Picardie Jules Verne MIS F-80000 Amiens France Univ Rouen Normandie LITIS F-76000 Rouen France Univ Cote dAzur CNRS I3S F-06900 Sophia Antipolis France

As part of the development of many robotic systems for the forestry sector, forest scene understanding requires the use of computer vision algorithms. However, this dense and unstructured environment is complex and puts conventional detection approaches to the test. In the case of tree instance segmentation, the presence of closely spaced or even intertwined trees, their highly variable shapes, and complex masks due to their branches and leaves are just some of the challenges to be overcome. For this, specific learning of tree boundaries is required to better distinguish one from another. In this letter, we propose ConvexMask, a convolutional neural network for real-time instance segmentation. ConvexMask opts for a label representation approach with a convex exterior polygon, defined by tree extremities, and a binary mask to handle the detail and occlusions that the label may contain. Experiments conducted on the SynthTree43k dataset show that ConvexMask distinguishes tree extremities better than state-of-the-art networks, resulting in better-quality masks.

关键词： deep learning for visual perception robotics and automation in agriculture and forestry

来源：评论

学校读者我要写书评

暂无评论

deep Fusion for Multi-Modal 6D Pose Estimation

引用

IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING 2024年第4期21卷 6540-6549页

作者： Lin, Shifeng Wang, Zunran Zhang, Shenghao Ling, Yonggen Yang, Chenguang South China Univ Technol Sch Automat Sci & Engn Key Lab Autonomous Syst & Networked Control Guangzhou 510640 Peoples R China Tencent Robot X Shenzhen 518057 Peoples R China Univ West England Bristol Robot Lab Bristol BS16 1QY England

6D pose estimation with individual modality encounters difficulties due to the limitations of modalities, such as RGB information on textureless objects and depth on reflective objects. This can be improved by exploiting the complementarity between modalities. Most of the previous methods only consider the correspondence between point clouds and RGB images and directly extract the features of the corresponding two modalities for fusion, which ignore the information of the modality itself and are negatively affected by erroneous background information when introducing more features for fusion. To enhance the complementarities between multiple modalities, we propose a neighbor-based cross-modalities attention mechanism for multi-modal 6D pose estimation. Neighbors represent that the RGB features of multiple neighbor are applied for fusion, which expands the receptive field. The cross-modalities attention mechanism leverages the similarities between the different modal features to help modal feature fusion, which reduces the negative impact of incorrect background information. Moreover, we design some features between the rendered image and the original image to obtain the confidence of pose estimation results. Experimental results on LM, LM-O and YCB-V datasets demonstrate the effectiveness of our methods. Video is available at https://***/watch?v=ApNBcX6NEGs.

关键词： Pose estimation Point cloud compression Feature extraction Decoding Fuses Solid modeling Encoding deep learning for visual perception RGB-D perception pose estimation

来源：评论

学校读者我要写书评

暂无评论

Do You Know the Way? Human-in-The-Loop Understanding for Fast Traversability Estimation in Mobile Robotics

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2025年第6期10卷 5863-5870页

作者： Schreiber, Andre Driggs-Campbell, Katherine Univ Illinois Dept Elect & Comp Engn Urbana IL 61820 USA Univ Illinois Coordinated Sci Lab Champaign IL 61820 USA

The increasing use of robots in unstructured environments necessitates the development of effective perception and navigation strategies to enable field robots to successfully perform their tasks. In particular, it is key for such robots to understand where in their environment they can and cannot travel-a task known as traversability estimation. However, existing geometric approaches to traversability estimation may fail to capture nuanced representations of traversability, whereas vision-based approaches typically either involve manually annotating a large number of images or require robot experience. In addition, existing methods can struggle to address domain shifts as they typically do not learn during deployment. To this end, we propose a human-in-the-loop (HiL) method for traversability estimation that prompts a human for annotations as-needed. Our method uses a foundation model to enable rapid learning on new annotations and to provide accurate predictions even when trained on a small number of quickly-provided HiL annotations. We extensively validate our method in simulation and on real-world data, and demonstrate that it can provide state-of-the-art traversability prediction performance.

关键词： Robots Navigation Annotations Estimation visualization Robot kinematics Training Predictive models Neural networks Laser radar Field robots vision-based navigation deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

EV-TTC: Event-Based Time to Collision Under Low Light Conditions

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2025年第6期10卷 6151-6158页

作者： Bisulco, Anthony Kumar, Vijay Daniilidis, Kostas Univ Penn GRASP Lab Philadelphia PA 19104 USA Athena RC Archimedes Unit Maroussi 15125 Greece

Rapid and accurate dense time-to-collision (TTC) estimation in resource-constrained, low-light environments is challenging for event-based camera systems. Fixed-time event representations like voxel grids face an inherent trade-off: larger temporal windows improve perception accuracy but increase storage demands, while smaller windows reduce storage at the cost of accuracy. We present a hardware-aware TTC estimation system designed for mobile robots, satisfying strict bandwidth, computation, and storage requirements. Our core innovation is a time-scale separation method for computing a multi-temporal scale event representation, achieving a latency of 3.3 ms at 75 Million Events per Second (MEPS). As part of this study, we developed Time-To-Collision/Event Flow ((TCEF)-C-2) a new high-temporal-resolution TTC dataset, using HD event cameras, with temporal estimates at least 7 times greater than existing event datasets such as MVSEC, DSEC, and VECtor via SE(3) interpolation. Our method outperforms existing approaches, reducing mean frame median TTC error by at least 20% compared to voxel grids on (TCEF)-C-2, and achieving an average 31% improvement over other baselines across multiple datasets. Our system runs in real-time on a Jetson Orin NX with just 9.5 ms latency at 141 Hz, outperforming all other methods on embedded hardware, making it ideal for mobile robots.

关键词： Estimation Event detection Cameras Optical flow IIR filters Finite impulse response filters Accuracy Optical filters Translation Robot vision systems deep learning for visual perception engineering for robotic systems visual learning

来源：评论

学校读者我要写书评

暂无评论

Efficient and accurate stereo matching via guided deformable aggregation

引用

JOURNAL OF ELECTRONIC IMAGING 2025年第2期34卷

作者： Li, Jie Li, Xinjia Chang, Mingyuan Wang, Lin Du, Shuangli Zhou, Jie Liu, Yiguang Shanxi Univ Finance & Econ Sch Informat Taiyuan Peoples R China Xian Univ Technol Sch Comp Sci & Engn Xian Peoples R China Sichuan Univ Sch Math Chengdu Peoples R China Sichuan Univ Sch Comp Sci Chengdu Peoples R China

In stereo vision, depth information relies on the dense registration accuracy of binocular stereo images, and its real-time performance is also significant in many automation applications. Recently, it is still a challenge to balance efficiency and accuracy. Motivated by this problem, we propose a lightweight two-dimensional (2D) guided deformable aggregation (GDA) module. It uses color prior information to learn the aggregation sampling points for fitting the irregular window, and it enables the fast recovery of lost high-frequency detail information from a coarse cost volume. Furthermore, we propose a guided deformable aggregation-based stereo matching network (GDANet) for balancing the efficiency and accuracy. It builds a fast three-dimensional network to obtain the cost volume of low-frequency non-detail regions and then uses the lightweight 2D GDA module to recover high-frequency detail regions. Experiments show that GDANet achieves better results than current high-efficiency methods in the Scene Flow and KITTI datasets. Especially, in edge regions and thin structures, our method shows better qualitative and quantitative results.

关键词： computer vision for automation deep learning for visual perception guided deformable aggregation

来源：评论

学校读者我要写书评

暂无评论

Spatial-Aware Dynamic Lightweight Self-Supervised Monocular Depth Estimation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第1期9卷 883-890页

作者： Song, Linna Shi, Dianxi Xia, Jianqiang Ouyang, Qianying Qiao, Ziteng Jin, Songchang Yang, Shaowu Natl Univ Def Technol Coll Comp Changsha 410073 Peoples R China Intelligent Game & Decis Lab Beijing 100091 Peoples R China Natl Innovat Inst Def Technol Beijing 100071 Peoples R China

Self-supervised monocular depth estimation has attracted extensive attention in recent years. Lightweight depth estimation methods are crucial for resource-constrained edge devices. However, existing lightweight methods often encounter the challenge of limited representation capacity and increased computational resource consumption for image reconstruction. To alleviate these issues, we propose a novel spatial-aware dynamic lightweight monocular depth estimation method (SAD-Depth). Specifically, we propose a spatial-aware dynamic encoder, which can capture spatial information of the input and generate input-adaptive dynamic convolutions, thereby significantly enhancing the model's adaptability to complex scenes. Meanwhile, we propose a multi-scale sub-pixel lightweight decoder that generates high-quality depth maps while maintaining a lightweight design. Experimental results demonstrate that our proposed SAD-Depth exhibits superiority in both model size and inference speed, achieving state-of-the-art performance on the KITTI benchmark.

关键词： deep learning for visual perception mapping lightweight methods

来源：评论

学校读者我要写书评

暂无评论

MA-Stereo: Real-Time Stereo Matching via Multi-Scale Attention Fusion and Spatial Error-Aware Refinement

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第11期9卷 9954-9961页

作者： Gao, Wei Cai, Yongjie Akoudad, Youssef Yang, Yang Chen, Jie Jiangsu Univ Sch Comp Sci & Telecommun Engn Zhenjiang 212013 Peoples R China Northwestern Polytech Univ CIAIC Sch Marine Sci & Technol Xian 710072 Peoples R China

Stereo matching is a fundamental task in computer vision. Real-time stereo matching has recently shown great potential in robotics and autonomous driving applications. However, the existing cost aggregation in real-time stereo matching suffers from accuracy limitations in ill-posed regions. Furthermore, most real-time stereo matching methods struggle to predict disparity in object details and edge areas, resulting in relatively blurred and lacking detailed disparity maps. To address these issues, we propose a real-time stereo matching architecture called MA-Stereo, which features a multi-scale attention fusion (MAF) module and an attention-based spatial error-aware refinement (ASER) module. The MAF adaptively fuses context and geometry information through attention mechanism, effectively improving cost aggregation. In addition, the ASER refines the predicted disparity map, fully leveraging high-frequency information and spatial evidence to accurately predict disparities for sharp edges and thin structures. Experimental results on the SceneFlow and KITTI benchmarks demonstrate that MA-Stereo outperforms almost all current state-of-the-art real-time stereo matching methods while maintaining relatively low runtime, achieving a favorable trade-off between accuracy and speed.

关键词： deep learning for visual perception autonomous vehicle navigation real-time stereo matching real-time stereo matching autonomous vehicle navigation real-time stereo matching

来源：评论

学校读者我要写书评

暂无评论

CoPeD-Advancing Multi-Robot Collaborative perception: A Comprehensive Dataset in Real-World Environments

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第7期9卷 6416-6423页

作者： Zhou, Yang Quang, Long Nieto-Granda, Carlos Loianno, Giuseppe NYU Tandon Sch Engn Brooklyn NY 11201 USA US Army Combat Capabil Dev Command Army Res Lab Adelphi MD 20783 USA

In the past decade, although single-robot perception has made significant advancements, the exploration of multi-robot collaborative perception remains largely unexplored. This involves fusing compressed, intermittent, limited, heterogeneous, and asynchronous environmental information across multiple robots to enhance overall perception, despite challenges like sensor noise, occlusions, and sensor failures. One major hurdle has been the lack of real-world datasets. This letter presents a pioneering and comprehensive real-world multi-robot collaborative perception dataset to boost research in this area. Our dataset leverages the untapped potential of air-ground robot collaboration featuring distinct spatial viewpoints, complementary robot mobilities, coverage ranges, and sensor modalities. It features raw sensor inputs, pose estimation, and optional high-level perception annotation, thus accommodating diverse research interests. Compared to existing datasets predominantly designed for Simultaneous Localization and Mapping (SLAM), our setup ensures a diverse range and adequate overlap of sensor views to facilitate the study of multi-robot collaborative perception algorithms. We demonstrate the value of this dataset qualitatively through multiple collaborative perception tasks. We believe this work will unlock the potential research of high-level scene understanding through multi-modal collaborative perception in multi-robot settings.

关键词： Data sets for robotic vision deep learning for visual perception multi-robot systems

来源：评论

学校读者我要写书评

暂无评论

Inverse Kinematics Embedded Network for Robust Patient Anatomy Avatar Reconstruction From Multimodal Data

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第4期9卷 3395-3402页

作者： Zhou, Tongxi Chen, Mingcong Cao, Guanglin Hu, Jian Liu, Hongbin Univ Chinese Acad Sci Sch Artificial Intelligence Beijing 100049 Peoples R China Chinese Acad Sci Inst Automat State Key Lab Multimodal Artificial Intelligence S Beijing 100190 Peoples R China City Univ Hong Kong Dept Biomed Engn Hong Kong Peoples R China Chinese Acad Sci Hong Kong Inst Sci & Innovat Ctr Artificial Intelligence & Robot Hong Kong Peoples R China Kings Coll London Sch Biomed Engn & Imaging Sci London SE1 7EU England

Patient modelling has a wide range of applications in medicine and healthcare, such as clinical teaching, surgery navigation and automatic robotized scanning. While patients are typically covered or occluded in medical scenes, directly regressing human meshes from single RGB images is challenging. To this end, we design a deep learning-based patient anatomy reconstruction network from RGB-D images with three key modules: 1) the attention-based multimodal fusion module, 2) the analytical inverse kinematics module and 3) the anatomical layer module. In our pipeline, the color and depth modality are fully fused by the multimodal attention module to obtain a cover-insensitive feature map. The estimated 3D keypoints, learned from the fused feature, are further converted to patient model parameters through the embedded analytical inverse kinematics module. To capture more detailed patient structures, we also present a parametric anatomy avatar by extending the Skinned Multi-Person Linear Model (SMPL) with internal bone and artery models. Final meshes are driven by the predicted parameters via the anatomical layer module, generating digital twins of patients. Experimental results on the Simultaneously-Collected Multimodal Lying Pose Dataset demonstrate that our approach surpasses state-of-the-art human mesh recovery methods and shows robustness to occlusions.

关键词： Image reconstruction Kinematics Three-dimensional displays Image color analysis Biomedical imaging Avatars Solid modeling Gesture posture and facial expressions deep learning for visual perception modeling and simulating humans RGB-D perception

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：