检索结果-内蒙古大学图书馆

OMEGA: Efficient Occlusion-Aware Navigation for Air-Ground Robots in Dynamic Environments via State Space Model

IEEE ROBOTICS AND AUTOMATION LETTERS 2025年第2期10卷 1066-1073页

作者： Wang, Junming Guan, Xiuxian Sun, Zekai Shen, Tianxiang Huang, Dong Liu, Fangming Cui, Heming Univ Hong Kong Hong Kong Peoples R China Huazhong Univ Sci & Technol Peng Cheng Lab Wuhan 430074 Peoples R China Shanghai AI Labolatory Shanghai 200232 Peoples R China

Air-ground robots (AGRs) are widely used in surveillance and disaster response due to their exceptional mobility and versatility (i.e., flying and driving). Current AGR navigation systems perform well in static occlusion-prone environments (e.g., indoors) by using 3D semantic occupancy networks to predict occlusions for complete local mapping and then computing Euclidean Signed Distance Field (ESDF) for path planning. However, these systems face challenges in dynamic scenes (e.g., crowds) due to limitations in perception networks' low prediction accuracy and path planners' high computation overhead. In this paper, we propose Omega, which contains OccMamba with an Efficient AGR-plAnner to address the above-mentioned problems. OccMamba adopts a novel architecture that separates semantic and occupancy prediction into independent branches, incorporating the mamba module to efficiently extract semantic and geometric features in 3D environments. This ensures the network can learn long-distance dependencies and improve prediction accuracy. Features are then combined within the Bird's Eye View (BEV) space to minimise computational overhead during feature fusion. The resulting semantic occupancy map is integrated into the local map, providing occlusion awareness of the dynamic environment. Our AGR-Planner utilizes this local map and employs Kinodynamic A* search and gradient-based trajectory optimization for ESDF-free and energy-efficient planning. Experiments demonstrate that OccMamba outperforms the state-of-the-art 3D semantic occupancy network with 25.0% mIoU. End-to-end navigation experiments in dynamic scenes verify OMEGA's efficiency, achieving a 96% average planning success rate.

关键词： Semantics Three-dimensional displays Navigation Accuracy Real-time systems Robots Planning Point cloud compression Energy efficiency Transformers deep learning for visual perception autonomous navigation 3D semantic occupancy prediction

来源：评论

学校读者我要写书评

暂无评论

Semi-supervised generative adversarial network to estimate the depth map of underwater targets in acoustic images for 3D reconstruction

引用

MEASUREMENT SCIENCE AND TECHNOLOGY 2025年第3期36卷 035403-035403页

作者： Bai, Boming Yan, Weisheng Cui, Rongxin Northwestern Polytech Univ Sch Marine Sci & Technol Xian 710072 Peoples R China

Owing to the unique imaging formulation principle of acoustic cameras, certain dimensions of the target may be lost in acoustic images. Some studies have been conducted to retrieve these missing details from a single pseudo front depth that contains 3D information about the target. Nevertheless, the use of a single pseudo front depth may face ambiguity problems in recovering the 3D information of acoustic targets. Therefore, this paper presents a novel semi-supervised generative adversarial network (SGAN) for estimating the depth of two adjacent acoustic images at know relative sonar poses, which generates high-accuracy reconstruction results, while reducing the dependence on the labeled ground truth of acoustic targets. Specifically, we design a semi-supervised depth estimation framework that utilizes image warping for semi-supervised training, resulting in denser depth maps. Various datasets containing underwater targets and terrain were generated to simulate real underwater scenarios. Experiments with simulator data and real images verify the feasibility of using SGAN for depth estimation in 3D target reconstruction.

关键词： generative adversarial network (GAN) acoustic camera-based 3D reconstruction deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

Towards Better Data Exploitation in Self-Supervised Monocular Depth Estimation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第1期9卷 763-770页

作者： Liu, Jinfeng Kong, Lingtong Yang, Jie Liu, Wei Shanghai Jiao Tong Univ Dept Automat Shanghai 200240 Peoples R China

Depth estimation plays an important role in robotic perception systems. The self-supervised monocular paradigm has gained significant attention since it can free training from the reliance on depth annotations. Despite recent advancements, existing self-supervised methods still underutilize the available training data, limiting their generalization ability. In this letter, we take two data augmentation techniques, namely Resizing-Cropping and Splitting-Permuting, to fully exploit the potential of training datasets. Specifically, the original image and the generated two augmented images are fed into the training pipeline simultaneously and we leverage them to conduct self-distillation. Additionally, we introduce the detail-enhanced DepthNet with an extra full-scale branch in the encoder and a grid decoder to enhance the restoration of fine details in depth maps. Experimental results demonstrate our method can achieve state-of-the-art performance on the KITTI and Cityscapes datasets. Moreover, our KITTI models also show superior generalization performance when transferring to Make3D, NYUv2 and Cityscapes datasets.

关键词： deep learning for visual perception deep learning methods visual learning

来源：评论

学校读者我要写书评

暂无评论

Co-Occ: Coupling Explicit Feature Fusion With Volume Rendering Regularization for Multi-Modal 3D Semantic Occupancy Prediction

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第6期9卷 5687-5694页

作者： Pan, Jingyi Wang, Zipeng Wang, Lin Hong Kong Univ Sci & Technol AI Thrust Guangzhou 511458 Guangdong Peoples R China Hong Kong Univ Sci & Technol AI CMA Thrust Guangzhou 511458 Guangdong Peoples R China Hong Kong Univ Sci & Technol Dept CSE Hong Kong Peoples R China

3D semantic occupancy prediction is a pivotal task in the field of autonomous driving. Recent approaches have made great advances in 3D semantic occupancy predictions on a single modality. However, multi-modal semantic occupancy prediction approaches have encountered difficulties in dealing with the modality heterogeneity, modality misalignment, and insufficient modality interactions that arise during the fusion of different modalities data, which may result in the loss of important geometric and semantic information. This letter presents a novel multi-modal, i.e., LiDAR-camera 3D semantic occupancy prediction framework, dubbed Co-Occ, which couples explicit LiDAR-camera feature fusion with implicit volume rendering regularization. The key insight is that volume rendering in the feature space can proficiently bridge the gap between 3D LiDAR sweeps and 2D images while serving as a physical regularization to enhance LiDAR-camera fused volumetric representation. Specifically, we first propose a Geometric- and Semantic-aware Fusion (GSFusion) module to explicitly enhance LiDAR features by incorporating neighboring camera features through a K-nearest neighbors (KNN) search. Then, we employ volume rendering to project the fused feature back to the image planes for reconstructing color and depth maps. These maps are then supervised by input images from the camera and depth estimations derived from LiDAR, respectively. Extensive experiments on the popular nuScenes and SemanticKITTI benchmarks verify the effectiveness of our Co-Occ for 3D semantic occupancy prediction.

关键词： Laser radar Three-dimensional displays Semantics Cameras Rendering (computer graphics) Feature extraction Task analysis deep learning for visual perception sensor fusion semantic scene understanding

来源：评论

学校读者我要写书评

暂无评论

Collaborative Multi-Object Tracking With Conformal Uncertainty Propagation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第4期9卷 3323-3330页

作者： Su, Sanbao Han, Songyang Li, Yiming Zhang, Zhili Feng, Chen Ding, Caiwen Miao, Fei Univ Connecticut Dept Comp Sci & Engn Storrs CT 06268 USA Univ Connecticut Storrs CT 06268 USA NYU Tandon Sch Engn Brooklyn NY 11201 USA

Object detection and multiple object tracking (MOT) are essential components of self-driving systems. Accurate detection and uncertainty quantification are both critical for onboard modules, such as perception, prediction, and planning, to improve the safety and robustness of autonomous vehicles. Collaborative object detection (COD) has been proposed to improve detection accuracy and reduce uncertainty by leveraging the viewpoints of multiple agents. However, little attention has been paid to how to leverage the uncertainty quantification from COD to enhance MOT performance. In this letter, as the first attempt to address this challenge, we design an uncertainty propagation framework called MOT-CUP. Our framework first quantifies the uncertainty of COD through direct modeling and conformal prediction, and propagates this uncertainty information into the motion prediction and association steps. MOT-CUP is designed to work with different collaborative object detectors and baseline MOT algorithms. We evaluate MOT-CUP on V2X-Sim, a comprehensive collaborative perception dataset, and demonstrate a 2% improvement in accuracy and a 2.67 x reduction in uncertainty compared to the baselines, e.g. SORT and ByteTrack. In scenarios characterized by high occlusion levels, our MOT-CUP demonstrates a noteworthy 4.01% improvement in accuracy. MOT-CUP demonstrates the importance of uncertainty quantification in both COD and MOT, and provides the first attempt to improve the accuracy and reduce the uncertainty in MOT based on COD through uncertainty propagation.

关键词： Uncertainty Object detection Collaboration Kalman filters Prediction algorithms Predictive models Estimation Computer vision for transportation deep learning for visual perception multi-robot systems visual tracking

来源：评论

学校读者我要写书评

暂无评论

learning Rearrangement Manipulation via Scene Prediction in Point Cloud

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第12期9卷 11090-11097页

作者： Ma, Anji Duan, Xingguang Beijing Inst Technol Sch Mechatron Engn Beijing 100081 Peoples R China

Predicting scene evolution conditioned on robotic actions is a vital technique in modeling robot manipulations. Previous studies have primarily focused on learning spatiotemporally continuous actions like Cartesian displacements, thus applying them to planar-pushing tasks. In this letter, we propose a scene prediction model that learns higher-level robot actions, such as grasping and pick-and-place, and is applicable to planning such actions in rearrangement manipulations. The model takes partially observed point clouds (e.g., from a single camera) and robot pick-and-place actions to predict point clouds of future scenes. The model directly learns scene prediction using point cloud observation and representation, without requiring prior knowledge of object properties like cad models, pose, or instance segmentation. We train the model only on a synthetic dataset acquired entirely automatically with minimal human intervention. Our experiments validate that our model can substantially learn robot grasping and pick-and-place actions and show that, when integrated into a sample-based planning framework, predicting scenes in point clouds outperforms the image-based baseline in grasping and rearrangement manipulation. Moreover, results show that our method can be directly transferred to real-world environments without fine-tuning and show promising performance on a collection of 18 household objects.

关键词： deep learning in grasping and manipulation manipulation planning deep learning for visual perception deep learning in grasping and manipulation manipulation planning

来源：评论

学校读者我要写书评

暂无评论

learning-To-Rank Approach for Identifying Everyday Objects Using a Physical-World Search Engine

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第3期9卷 2088-2095页

作者： Kaneda, Kanta Nagashima, Shunya Korekata, Ryosuke Kambara, Motonari Sugiura, Komei Keio Univ Yokohama 2238522 Japan

Domestic service robots offer a solution to the increasing demand for daily care and support. A human-in-the-loop approach that combines automation and operator intervention is considered to be a realistic approach to their use in society. Therefore, we focus on the task of retrieving target objects from open-vocabulary user instructions in a human-in-the-loop setting, which we define as the learning-to-rank physical objects (LTRPO) task. For example, given the instruction "Please go to the dining room which has a round table. Pick up the bottle on it," the model is required to output a ranked list of target objects that the operator/user can select. In this letter, we propose MultiRankIt, which is a novel approach for the LTRPO task. MultiRankIt introduces the Crossmodal Noun Phrase Encoder to model the relationship between phrases that contain referring expressions and the target bounding box, and the Crossmodal Region Feature Encoder to model the relationship between the target object and multiple images of its surrounding contextual environment. Additionally, we built a new dataset for the LTRPO task that consists of instructions with complex referring expressions accompanied by real indoor environmental images that feature various target objects. We validated our model on the dataset and it outperformed the baseline method in terms of the mean reciprocal rank and recall@k. Furthermore, we conducted physical experiments in a setting where a domestic service robot retrieved everyday objects in a standardized domestic environment, based on users' instruction in a human-in-the-loop setting. The experimental results demonstrate that the success rate for object retrieval achieved 80%.

关键词： deep learning methods learning categories and concepts deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

DiffMap: Enhancing Map Segmentation With Map Prior Using Diffusion Model

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第11期9卷 9836-9843页

作者： Jia, Peijin Wen, Tuopu Luo, Ziang Yang, Mengmeng Jiang, Kun Liu, Ziyuan Tang, Xuewei Lei, Zhiquan Cui, Le Zhang, Bo Sheng, Kehua Yang, Diange Tsinghua Univ Sch Vehicle & Mobil Beijing 100084 Peoples R China DiDi Chuxing Tianjin 300450 Peoples R China

Constructing high-definition (HD) maps is a crucial requirement for enabling autonomous driving. In recent years, several map segmentation algorithms have been developed to address this need, leveraging advancements in Bird's-Eye View (BEV) perception. However, existing models still encounter challenges in producing realistic and consistent semantic map layouts. A prominent issue is the limited utilization of structured priors inherent in map segmentation masks. In light of this, we propose DiffMap, a novel approach specifically designed to model the structured priors of map segmentation masks using latent diffusion model. By incorporating this technique, the performance of existing semantic segmentation methods can be significantly enhanced and certain structural errors present in the segmentation outputs can be effectively rectified. Notably, the proposed module can be seamlessly integrated into any map segmentation model, thereby augmenting its capability to accurately delineate semantic information. Furthermore, through extensive visualization analysis, our model demonstrates superior proficiency in generating results that more accurately reflect real-world map layouts, further validating its efficacy in improving the quality of the generated maps.

关键词： Semantics Diffusion models Noise reduction Predictive models Laser radar Standards Noise Autonomous vehicle navigation deep learning for visual perception sensor fusion

来源：评论

学校读者我要写书评

暂无评论

SE3ET: SE(3)-Equivariant Transformer for Low-Overlap Point Cloud Registration

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第11期9卷 9526-9533页

作者： Lin, Chien Erh Zhu, Minghan Ghaffari, Maani Univ Michigan Ann Arbor MI 48109 USA

Partial point cloud registration is a challenging problem in robotics, especially when the robot undergoes a large transformation, causing a significant initial pose error and a low overlap between measurements. This letter proposes exploiting equivariant learning from 3D point clouds to improve registration robustness. We propose SE3ET, an SE(3)-equivariant registration framework that employs equivariant point convolution and equivariant transformer designs to learn expressive and robust geometric features. We tested the proposed registration method on indoor and outdoor benchmarks where the point clouds are under arbitrary transformations and low overlapping ratios. We also provide generalization tests and run-time performance.

关键词： Point cloud compression Transformers Feature extraction Three-dimensional displays Representation learning Convolution Decoding deep learning for visual perception localization deep learning methods

来源：评论

学校读者我要写书评

暂无评论

AiSDF: Structure-Aware Neural Signed Distance Fields in Indoor Scenes

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第5期9卷 4106-4113页

作者： Jang, Jaehoon Lee, Inha Kim, Minje Joo, Kyungdon UNIST Artificial Intelligence Grad Sch Ulsan South Korea UNIST Dept Comp Sci & Engn Ulsan South Korea

Indoor scenes we are living in are visually homogenous or textureless, while they inherently have structural forms and provide enough structural priors for 3D scene reconstruction. Motivated by this fact, we propose a structure-aware online signed distance fields (SDF) reconstruction framework in indoor scenes, especially under the Atlanta world (AW) assumption. Thus, we dub this incremental SDF reconstruction for AW as AiSDF. Within the online framework, we infer the underlying Atlanta structure of a given scene and then estimate planar surfel regions supporting the Atlanta structure. This Atlanta-aware surfel representation provides an explicit planar map for a given scene. In addition, based on these Atlanta planar surfel regions, we adaptively sample and constrain the structural regularity in the SDF reconstruction, which enables us to improve the reconstruction quality by maintaining a high-level structure while enhancing the details of a given scene. We evaluate the proposed AiSDF on the ScanNet and ReplicaCAD datasets, where we demonstrate that the proposed framework is capable of reconstructing fine details of objects implicitly, as well as structures explicitly in room-scale scenes.

关键词： deep learning for visual perception mapping incremental learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：