检索结果-内蒙古大学图书馆

DiffMap: Enhancing Map Segmentation With Map Prior Using Diffusion Model

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第11期9卷 9836-9843页

作者： Jia, Peijin Wen, Tuopu Luo, Ziang Yang, Mengmeng Jiang, Kun Liu, Ziyuan Tang, Xuewei Lei, Zhiquan Cui, Le Zhang, Bo Sheng, Kehua Yang, Diange Tsinghua Univ Sch Vehicle & Mobil Beijing 100084 Peoples R China DiDi Chuxing Tianjin 300450 Peoples R China

Constructing high-definition (HD) maps is a crucial requirement for enabling autonomous driving. In recent years, several map segmentation algorithms have been developed to address this need, leveraging advancements in Bird's-Eye View (BEV) perception. However, existing models still encounter challenges in producing realistic and consistent semantic map layouts. A prominent issue is the limited utilization of structured priors inherent in map segmentation masks. In light of this, we propose DiffMap, a novel approach specifically designed to model the structured priors of map segmentation masks using latent diffusion model. By incorporating this technique, the performance of existing semantic segmentation methods can be significantly enhanced and certain structural errors present in the segmentation outputs can be effectively rectified. Notably, the proposed module can be seamlessly integrated into any map segmentation model, thereby augmenting its capability to accurately delineate semantic information. Furthermore, through extensive visualization analysis, our model demonstrates superior proficiency in generating results that more accurately reflect real-world map layouts, further validating its efficacy in improving the quality of the generated maps.

关键词： Semantics Diffusion models Noise reduction Predictive models Laser radar Standards Noise Autonomous vehicle navigation deep learning for visual perception sensor fusion

来源：评论

学校读者我要写书评

暂无评论

SE3ET: SE(3)-Equivariant Transformer for Low-Overlap Point Cloud Registration

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第11期9卷 9526-9533页

作者： Lin, Chien Erh Zhu, Minghan Ghaffari, Maani Univ Michigan Ann Arbor MI 48109 USA

Partial point cloud registration is a challenging problem in robotics, especially when the robot undergoes a large transformation, causing a significant initial pose error and a low overlap between measurements. This letter proposes exploiting equivariant learning from 3D point clouds to improve registration robustness. We propose SE3ET, an SE(3)-equivariant registration framework that employs equivariant point convolution and equivariant transformer designs to learn expressive and robust geometric features. We tested the proposed registration method on indoor and outdoor benchmarks where the point clouds are under arbitrary transformations and low overlapping ratios. We also provide generalization tests and run-time performance.

关键词： Point cloud compression Transformers Feature extraction Three-dimensional displays Representation learning Convolution Decoding deep learning for visual perception localization deep learning methods

来源：评论

学校读者我要写书评

暂无评论

AiSDF: Structure-Aware Neural Signed Distance Fields in Indoor Scenes

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第5期9卷 4106-4113页

作者： Jang, Jaehoon Lee, Inha Kim, Minje Joo, Kyungdon UNIST Artificial Intelligence Grad Sch Ulsan South Korea UNIST Dept Comp Sci & Engn Ulsan South Korea

Indoor scenes we are living in are visually homogenous or textureless, while they inherently have structural forms and provide enough structural priors for 3D scene reconstruction. Motivated by this fact, we propose a structure-aware online signed distance fields (SDF) reconstruction framework in indoor scenes, especially under the Atlanta world (AW) assumption. Thus, we dub this incremental SDF reconstruction for AW as AiSDF. Within the online framework, we infer the underlying Atlanta structure of a given scene and then estimate planar surfel regions supporting the Atlanta structure. This Atlanta-aware surfel representation provides an explicit planar map for a given scene. In addition, based on these Atlanta planar surfel regions, we adaptively sample and constrain the structural regularity in the SDF reconstruction, which enables us to improve the reconstruction quality by maintaining a high-level structure while enhancing the details of a given scene. We evaluate the proposed AiSDF on the ScanNet and ReplicaCAD datasets, where we demonstrate that the proposed framework is capable of reconstructing fine details of objects implicitly, as well as structures explicitly in room-scale scenes.

关键词： deep learning for visual perception mapping incremental learning

来源：评论

学校读者我要写书评

暂无评论

Exploiting Depth Priors for Few-Shot Neural Radiance Field Reconstruction

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第11期9卷 9844-9851页

作者： Chen, Shuya Li, Zheyang Zhu, Hao Tan, Wenming Ren, Ye Xiang, Zhiyu Hikvis Res Inst Hangzhou 310000 Peoples R China Zhejiang Univ Coll Informat Sci & Elect Engn Hangzhou 310000 Peoples R China Zhejiang Univ Zhejiang Prov Key Lab Informat Proc Commun & Netwo Hangzhou 310000 Peoples R China

The performance of neural radiance field technologies deteriorates rapidly when sparse views are used as input. In this paper, we propose a simulated viewpoint enhancement for surface reconstruction that extracts diverse geometric features from the depth to address this limitation. We design a novel pseudo-view generation approach to simulate dense viewpoints, in which the RGB-D pairs are converted into new images corresponding to the translational poses through reprojection. To address potential holes in the pseudo-views resulting from occlusion in the reprojecting process, we propose a hole handling mechanism to exclude them during training. Additionally, normal information is derived from the depth, serving as supervision to enhance the scene geometry. Experiments conducted on various baseline models and datasets demonstrate that our algorithm effectively utilizes the inherent geometry embedded in the depth maps, leading to a significant improvement in the quality of 3D surface reconstruction compared to existing methods.

关键词： Image reconstruction Point cloud compression Neural radiance field Surface reconstruction Training Image color analysis Rendering (computer graphics) deep learning for visual perception computer vision for automation surface reconstruction neural radiance field RGB-D

来源：评论

学校读者我要写书评

暂无评论

Mono-Camera-Only Target Chasing for a Drone in a Dense Environment by Cross-Modal learning

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第8期9卷 7254-7261页

作者： Yoo, Seungyeon Jung, Seungwoo Lee, Yunwoo Shim, Dongseok Kim, H. Jin Seoul Natl Univ Automat & Syst Res Inst Dept Aerosp Engn Seoul 08826 South Korea Seoul Natl Univ Artificial Intelligence Inst SNU Seoul 08826 South Korea Seoul Natl Univ Interdisciplinary Program AI Seoul 08826 South Korea

Chasing a dynamic target in a dense environment is one of the challenging applications of autonomous drones. The task requires multi-modal data, such as RGB and depth, to accomplish safe and robust maneuver. However, using different types of modalities can be difficult due to the limited capacity of drones in aspects of hardware complexity and sensor cost. Our framework resolves such restrictions in the target chasing task by using only a monocular camera instead of multiple sensor inputs. From an RGB input, the perception module can extract a cross-modal representation containing information from multiple data modalities. To learn cross-modal representations at training time, we employ variational autoencoder (VAE) structures and the joint objective function across heterogeneous data. Subsequently, using latent vectors acquired from the pre-trained perception module, the planning module generates a proper next-time-step waypoint by imitation learning of the expert, which performs a numerical optimization using the privileged RGB-D data. Furthermore, the planning module considers temporal information of the target to improve tracking performance through consecutive cross-modal representations. Ultimately, we demonstrate the effectiveness of our framework through the reconstruction results of the perception module, the target chasing performance of the planning module, and the zero-shot sim-to-real deployment of a drone.

关键词： Drones Task analysis Planning Target tracking Vehicle dynamics Training Image reconstruction Vision-based navigation visual learning deep learning for visual perception deep learning methods

来源：评论

学校读者我要写书评

暂无评论

KVN: Keypoints Voting Network With Differentiable RANSAC for Stereo Pose Estimation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第4期9卷 3498-3505页

作者： Donadi, Ivano Pretto, Alberto Univ Padua Dept Informat Engn I-35131 Padua Italy

Object pose estimation is a fundamental computer vision task exploited in several robotics and augmented reality applications. Many established approaches rely on predicting 2D-3D keypoint correspondences using RANSAC (Random sample consensus) and estimating the object pose using the PnP (Perspective-n-Point) algorithm. Being RANSAC non-differentiable, correspondences cannot be directly learned in an end-to-end fashion. In this letter, we address the stereo image-based object pose estimation problem by i) introducing a differentiable RANSAC layer into a well-known monocular pose estimation network;ii) exploiting an uncertainty-driven multi-view PnP solver which can fuse information from multiple views. We evaluate our approach on a challenging public stereo object pose estimation dataset and a custom-built dataset we call Transparent Tableware Dataset (TTD), yielding state-of-the-art results against other recent approaches. Furthermore, in our ablation study, we show that the differentiable RANSAC layer plays a significant role in the accuracy of the proposed method. We release with this letter the code of our method and the TTD dataset.

关键词： Computer vision for automation deep learning for visual perception perception for grasping and manipulation

来源：评论

学校读者我要写书评

暂无评论

learning Keypoints for Robotic Cloth Manipulation Using Synthetic Data

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第7期9卷 6528-6535页

作者： Lips, Thomas De Gusseme, Victor-Louis Wyffels, Francis Ghent Univ Imec AI & Robot Lab IDLab AIRO B-9052 Zwijnaarde Belgium

Assistive robots should be able to wash, fold or iron clothes. However, due to the variety, deformability and self-occlusions of clothes, creating robot systems for cloth manipulation is challenging. Synthetic data is a promising direction to improve generalization, but the sim-to-real gap limits its effectiveness. To advance the use of synthetic data for cloth manipulation tasks such as robotic folding, we present a synthetic data pipeline to train keypoint detectors for almost-flattened cloth items. To evaluate its performance, we have also collected a real-world dataset. We train detectors for both T-shirts, towels and shorts and obtain an average precision of 64% and an average keypoint distance of 18 pixels. Fine-tuning on real-world data improves performance to 74% mAP and an average distance of only 9 pixels. Furthermore, we describe failure modes of the keypoint detectors and compare different approaches to obtain cloth meshes and materials. We also quantify the remaining sim-to-real gap and argue that further improvements to the fidelity of cloth assets will be required to further reduce this gap. The code, dataset and trained models are available here.

关键词： Robots Synthetic data Pipelines Detectors Semantics Flexible printed circuits Deformation Data sets for robotic vision deep learning for visual perception simulation and animation

来源：评论

学校读者我要写书评

暂无评论

Bathymetric Surveying With Imaging Sonar Using Neural Volume Rendering

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第9期9卷 8146-8153页

作者： Xie, Yiping Troni, Giancarlo Bore, Nils Folkesson, John KTH Royal Inst Technol Robot Percept & Learning Div S-10044 Stockholm Sweden Monterey Bay Aquarium Res Inst Moss Landing CA 95039 USA Ocean Infin SE-42671 Vastra Frolunda Sweden

This research addresses the challenge of estimating bathymetry from imaging sonars where the state-of-the-art works have primarily relied on either supervised learning with ground-truth labels or surface rendering based on the Lambertian assumption. In this letter, we propose a novel, self-supervised framework based on volume rendering for reconstructing bathymetry using forward-looking sonar (FLS) data collected during standard surveys. We represent the seafloor as a neural heightmap encapsulated with a parametric multi-resolution hash encoding scheme and model the sonar measurements with a differentiable renderer using sonar volumetric rendering employed with hierarchical sampling techniques. Additionally, we model the horizontal and vertical beam patterns and estimate them jointly with the bathymetry. We evaluate the proposed method quantitatively on simulation and field data collected by remotely operated vehicles (ROVs) during low-altitude surveys. Results show that the proposed method outperforms the current state-of-the-art approaches that use imaging sonars for seabed mapping. We also demonstrate that the proposed approach can potentially be used to increase the resolution of a low-resolution prior map with FLS data from low-altitude surveys.

关键词： Sonar Bathymetry Image reconstruction Three-dimensional displays Rendering (computer graphics) Surveys Encoding Bathymetric reconstruction deep learning methods deep learning for visual perception mapping marine robotics

来源：评论

学校读者我要写书评

暂无评论

RomniStereo: Recurrent Omnidirectional Stereo Matching

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第3期9卷 2511-2518页

作者： Jiang, Hualie Xu, Rui Tan, Minglang Jiang, Wenjie Insta360 Res Shenzhen 518172 Peoples R China

Omnidirectional stereo matching (OSM) is an essential and reliable means for $360{\circ }$ depth sensing. However, following earlier works on conventional stereo matching, prior state-of-the-art (SOTA) methods rely on... 详细信息

Omnidirectional stereo matching (OSM) is an essential and reliable means for $360<^>{\circ }$ depth sensing. However, following earlier works on conventional stereo matching, prior state-of-the-art (SOTA) methods rely on a 3D encoder-decoder block to regularize the cost volume, causing the whole system complicated and sub-optimal results. Recently, the Recurrent All-pairs Field Transforms (RAFT) based approach employs the recurrent update in 2D and has efficiently improved image-matching tasks, i.e., optical flow, and stereo matching. To bridge the gap between OSM and RAFT, we mainly propose an opposite adaptive weighting scheme to seamlessly transform the outputs of spherical sweeping of OSM into the required inputs for the recurrent update, thus creating a recurrent omnidirectional stereo matching (RomniStereo) algorithm. Furthermore, we introduce two techniques, i.e., grid embedding and adaptive context feature generation, which also contribute to RomniStereo's performance. Our best model improves the average MAE metric by 40.7% over the previous SOTA baseline across five datasets. When visualizing the results, our models demonstrate clear advantages on both synthetic and realistic examples.

关键词： Cameras Feature extraction Three-dimensional displays Costs Correlation Estimation Robot vision systems deep learning for visual perception omnidirectional vision stereo matching depth estimation

来源：评论

学校读者我要写书评

暂无评论

Rapid-Mapping: LiDAR-visual Implicit Neural Representations for Real-Time Dense Mapping

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第9期9卷 8154-8161页

作者： Zhang, Hanwen Zou, Yujie Yan, Zhewen Cheng, Hui Sun Yat Sen Univ Sch Comp Sci & Engn Guangzhou 510006 Peoples R China

Real-time dense mapping with high-fidelity textures in large-scale environments is such a challenge in robots, digital twins, and AR/VR applications. Neural Radiance Field (NeRF) has demonstrated remarkable capabilities in capturing intricate details and saving memory space, which provides significant advantages in the fine-grained reconstruction of large-scale scenes. Existing LiDAR-based mapping methods have not harnessed NeRF's ability to capture textures. In this letter, we propose the first real-time LiDAR-visual mapping method in large-scale indoor and outdoor environments, named Rapid-Mapping, that utilizes implicit neural representations and preserves high-fidelity textures. First of all, to align the camera image and LiDAR depth, we propose a method for extrinsic refinement to mitigate the issue of texture blurring caused by long-range and extended temporal scale measurements. Also, to alleviate the effects of lighting conditions and camera hardware interference, we utilize prior hue information to constrain the inverse affine transformation. Extensive experiments validate that Rapid-Mapping enables real-time dense mapping in large-scale complex indoor and outdoor scenes, exhibiting more detailed realistic textures and more accurate geometry compared to existing methods.

关键词： deep learning for visual perception mapping visual learning deep learning for visual perception mapping visual learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：