检索结果-内蒙古大学图书馆

Exploiting Depth Priors for Few-Shot Neural Radiance Field Reconstruction

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第11期9卷 9844-9851页

作者： Chen, Shuya Li, Zheyang Zhu, Hao Tan, Wenming Ren, Ye Xiang, Zhiyu Hikvis Res Inst Hangzhou 310000 Peoples R China Zhejiang Univ Coll Informat Sci & Elect Engn Hangzhou 310000 Peoples R China Zhejiang Univ Zhejiang Prov Key Lab Informat Proc Commun & Netwo Hangzhou 310000 Peoples R China

The performance of neural radiance field technologies deteriorates rapidly when sparse views are used as input. In this paper, we propose a simulated viewpoint enhancement for surface reconstruction that extracts diverse geometric features from the depth to address this limitation. We design a novel pseudo-view generation approach to simulate dense viewpoints, in which the RGB-D pairs are converted into new images corresponding to the translational poses through reprojection. To address potential holes in the pseudo-views resulting from occlusion in the reprojecting process, we propose a hole handling mechanism to exclude them during training. Additionally, normal information is derived from the depth, serving as supervision to enhance the scene geometry. Experiments conducted on various baseline models and datasets demonstrate that our algorithm effectively utilizes the inherent geometry embedded in the depth maps, leading to a significant improvement in the quality of 3D surface reconstruction compared to existing methods.

关键词： Image reconstruction Point cloud compression Neural radiance field Surface reconstruction Training Image color analysis Rendering (computer graphics) deep learning for visual perception computer vision for automation surface reconstruction neural radiance field RGB-D

来源：评论

学校读者我要写书评

暂无评论

MP6D: An RGB-D Dataset for Metal Parts' 6D Pose Estimation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第3期7卷 5912-5919页

作者： Chen, Long Yang, Han Wu, Chenrui Wu, Shiqing Univ Shanghai Sci & Technol Dept Mech Engn Shanghai 200093 Peoples R China

We introduce MP6D, a public dataset which is used for 6D pose estimation of Metal Parts in industrial environments. The dataset consists of 20 metal parts made of aluminum alloy material which are commonly used in factories. To the best of our knowledge, this is the first dataset of metal parts with simultaneous multi-target, occluded, and illumination changes. The color homogeneity, textureless and light-reflecting properties raise great challenges for estimating the pose of the objects. To improve the accuracy of the ground-truth pose, we propose a novel bi-directional optimization method to minimize the projection errors of all the objects in the dataset. Our main insight is to iteratively optimize the union pose of multi-object in single-frame and the relative pose of each single object with respect to the Aurco-board through multi-frame information. We compare our dataset with several well-known pose estimation datasets. The results demonstrate that the accuracy of our dataset is superior to the others. We also provide the baselines for some state-of-the-art pose estimation methods upon our dataset for the comparison of further studies.

关键词： deep learning for visual perception pose estimation RGB-D perception

来源：评论

学校读者我要写书评

暂无评论

AiSDF: Structure-Aware Neural Signed Distance Fields in Indoor Scenes

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第5期9卷 4106-4113页

作者： Jang, Jaehoon Lee, Inha Kim, Minje Joo, Kyungdon UNIST Artificial Intelligence Grad Sch Ulsan South Korea UNIST Dept Comp Sci & Engn Ulsan South Korea

Indoor scenes we are living in are visually homogenous or textureless, while they inherently have structural forms and provide enough structural priors for 3D scene reconstruction. Motivated by this fact, we propose a structure-aware online signed distance fields (SDF) reconstruction framework in indoor scenes, especially under the Atlanta world (AW) assumption. Thus, we dub this incremental SDF reconstruction for AW as AiSDF. Within the online framework, we infer the underlying Atlanta structure of a given scene and then estimate planar surfel regions supporting the Atlanta structure. This Atlanta-aware surfel representation provides an explicit planar map for a given scene. In addition, based on these Atlanta planar surfel regions, we adaptively sample and constrain the structural regularity in the SDF reconstruction, which enables us to improve the reconstruction quality by maintaining a high-level structure while enhancing the details of a given scene. We evaluate the proposed AiSDF on the ScanNet and ReplicaCAD datasets, where we demonstrate that the proposed framework is capable of reconstructing fine details of objects implicitly, as well as structures explicitly in room-scale scenes.

关键词： deep learning for visual perception mapping incremental learning

来源：评论

学校读者我要写书评

暂无评论

KVN: Keypoints Voting Network With Differentiable RANSAC for Stereo Pose Estimation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第4期9卷 3498-3505页

作者： Donadi, Ivano Pretto, Alberto Univ Padua Dept Informat Engn I-35131 Padua Italy

Object pose estimation is a fundamental computer vision task exploited in several robotics and augmented reality applications. Many established approaches rely on predicting 2D-3D keypoint correspondences using RANSAC (Random sample consensus) and estimating the object pose using the PnP (Perspective-n-Point) algorithm. Being RANSAC non-differentiable, correspondences cannot be directly learned in an end-to-end fashion. In this letter, we address the stereo image-based object pose estimation problem by i) introducing a differentiable RANSAC layer into a well-known monocular pose estimation network;ii) exploiting an uncertainty-driven multi-view PnP solver which can fuse information from multiple views. We evaluate our approach on a challenging public stereo object pose estimation dataset and a custom-built dataset we call Transparent Tableware Dataset (TTD), yielding state-of-the-art results against other recent approaches. Furthermore, in our ablation study, we show that the differentiable RANSAC layer plays a significant role in the accuracy of the proposed method. We release with this letter the code of our method and the TTD dataset.

关键词： Computer vision for automation deep learning for visual perception perception for grasping and manipulation

来源：评论

学校读者我要写书评

暂无评论

Mono-Camera-Only Target Chasing for a Drone in a Dense Environment by Cross-Modal learning

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第8期9卷 7254-7261页

作者： Yoo, Seungyeon Jung, Seungwoo Lee, Yunwoo Shim, Dongseok Kim, H. Jin Seoul Natl Univ Automat & Syst Res Inst Dept Aerosp Engn Seoul 08826 South Korea Seoul Natl Univ Artificial Intelligence Inst SNU Seoul 08826 South Korea Seoul Natl Univ Interdisciplinary Program AI Seoul 08826 South Korea

Chasing a dynamic target in a dense environment is one of the challenging applications of autonomous drones. The task requires multi-modal data, such as RGB and depth, to accomplish safe and robust maneuver. However, using different types of modalities can be difficult due to the limited capacity of drones in aspects of hardware complexity and sensor cost. Our framework resolves such restrictions in the target chasing task by using only a monocular camera instead of multiple sensor inputs. From an RGB input, the perception module can extract a cross-modal representation containing information from multiple data modalities. To learn cross-modal representations at training time, we employ variational autoencoder (VAE) structures and the joint objective function across heterogeneous data. Subsequently, using latent vectors acquired from the pre-trained perception module, the planning module generates a proper next-time-step waypoint by imitation learning of the expert, which performs a numerical optimization using the privileged RGB-D data. Furthermore, the planning module considers temporal information of the target to improve tracking performance through consecutive cross-modal representations. Ultimately, we demonstrate the effectiveness of our framework through the reconstruction results of the perception module, the target chasing performance of the planning module, and the zero-shot sim-to-real deployment of a drone.

关键词： Drones Task analysis Planning Target tracking Vehicle dynamics Training Image reconstruction Vision-based navigation visual learning deep learning for visual perception deep learning methods

来源：评论

学校读者我要写书评

暂无评论

visual-Tactile Cross-Modal Data Generation Using Residue-Fusion GAN With Feature-Matching and Perceptual Losses

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第4期6卷 7525-7532页

作者： Cai, Shaoyu Zhu, Kening Ban, Yuki Narumi, Takuji City Univ Hong Kong Sch Creat Media Hong Kong Peoples R China City Univ Hong Kong Shenzhen Res Inst Shenzhen Peoples R China Univ Tokyo Grad Sch Frontier Sci Chiba Japan Univ Tokyo Grad Sch Informat Sci & Technol Tokyo Japan JST PRESTO Tokyo Japan

Existing psychophysical studies have revealed that the cross-modal visual-tactile perception is common for humans performing daily activities. However, it is still challenging to build the algorithmic mapping from one modality space to another, namely the cross-modal visual-tactile data translation/generation, which could be potentially important for robotic operation. In this letter, we propose a deep-learning-based approach for cross-modal visual-tactile data generation by leveraging the framework of the generative adversarial networks (GANs). Our approach takes the visual image of a material surface as the visual data, and the accelerometer signal induced by the pen-sliding movement on the surface as the tactile data. We adopt the conditional-GAN (cGAN) structure together with the residue-fusion (RF) module, and train the model with the additional feature-matching (FM) and perceptual losses to achieve the cross-modal data generation. The experimental results show that the inclusion of the RF module, and the FM and the perceptual losses significantly improves cross-modal data generation performance in terms of the classification accuracy upon the generated data and the visual similarity between the ground-truth and the generated data.

关键词： visual learning deep learning for visual perception haptics and haptic interfaces

来源：评论

学校读者我要写书评

暂无评论

Active Visuo-Haptic Object Shape Completion

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第2期7卷 5254-5261页

作者： Rustler, Lukas Lundell, Jens Behrens, Jan Kristof Kyrki, Ville Hoffmann, Matej Czech Tech Univ Fac Elect Engn Dept Cybernet Prague 16636 Czech Republic Aalto Univ Sch Elect Engn Dept Elect Engn & Automat Intelligent Robot Grp Espoo 02150 Finland Czech Tech Univ Czech Inst Informat Robot & Cybernet Prague 16636 Czech Republic

Recent advancements in object shape completion have enabled impressive object reconstructions using only visual input. However, due to self-occlusion, the reconstructions have high uncertainty in the occluded object parts, which negatively impacts the performance of downstream robotic tasks such as grasping. In this letter, we propose an active visuo-haptic shape completion method called Act-VH that actively computes where to touch the objects based on the reconstruction uncertainty. Act-VH reconstructs objects from point clouds and calculates the reconstruction uncertainty using IGR, a recent state-of-the-art implicit surface deep neural network. We experimentally evaluate the reconstruction accuracy of Act-VH against five baselines in simulation and in the real world. We also propose a new simulation environment for this purpose. The results show that Act-VH outperforms all baselines and that an uncertainty-driven haptic exploration policy leads to higher reconstruction accuracy than a random policy and a policy driven by Gaussian Process Implicit Surfaces. As a final experiment, we evaluate Act-VH and the best reconstruction baseline on grasping 10 novel objects. The results show that Act-VH reaches a significantly higher grasp success rate than the baseline on all objects. Together, this letter opens up the door for using active visuo-haptic shape completion in more complex cluttered scenes.

关键词： deep learning for visual perception perception for grasping and manipulation RGB-D perception

来源：评论

学校读者我要写书评

暂无评论

State-Consistency Loss for learning Spatial perception Tasks From Partial Labels

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第2期6卷 1112-1119页

作者： Nava, Mirko Gambardella, Luca Maria Giusti, Alessandro Dalle Molle Inst Artificial Intelligence IDSIA USI SUPSI CH-6928 Lugano Switzerland

When learning models for real-world robot spatial perception tasks, one might have access only to partial labels: this occurs for example in semi-supervised scenarios (in which labels are not available for a subset of the training instances) or in some types of self-supervised robot learning (where the robot autonomously acquires a labeled training set, but only acquires labels for a subset of the output variables in each instance). We introduce a general approach to deal with this class of problems using an auxiliary loss enforcing the expectation that the perceived environment state should not abruptly change;then, we instantiate the approach to solve two robot perception problems: a simulated ground robot learning long-range obstacle mapping as a 400-binary-label classification task in a self-supervised way in a static environment;and a real nano-quadrotor learning human pose estimation as a 3-variable regression task in a semi-supervised way in a dynamic environment. In both cases, our approach yields significant quantitative performance improvements (average increase of 6 AUC percentage points in the former;relative improvement of the R-2 metric ranging from 7% to 33% in the latter) over baselines.

关键词： Robots Task analysis Robot sensing systems Training Sensors Predictive models Robot vision systems deep learning methods deep learning for visual perception developmental robotics

来源：评论

学校读者我要写书评

暂无评论

Lidar Upsampling With Sliced Wasserstein Distance

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2023年第1期8卷 392-399页

作者： Savkin, Artem Wang, Yida Wirkert, Sebastian Navab, Nassir Tombari, Federico Tech Univ Munich Sch Computat Informat & Technol Dept Comp Sci D-80333 Munich Germany BMW Grp D-80788 Munich Germany Johns Hopkins Univ Dept Comp Sci Baltimore MD 21218 USA Google CH-8002 Zurich Switzerland

Lidar became an important component of the perception systems in autonomous driving. But challenges of training data acquisition and annotation made emphasized the role of the sensor to sensor domain adaptation. In this letter, we address the problem of lidar upsampling. learning on lidar point clouds is rather a challenging task due to their irregular and sparse structure. Here we propose a method for lidar point cloud upsampling which can reconstruct fine-grained lidar scan patterns. The key idea is to utilize edge-aware dense convolutions for both feature extraction and feature expansion. Additionally applying a more accurate Sliced Wasserstein Distance facilitates learning of the fine lidar sweep structures. This in turn enables our method to employ a one-stage upsampling paradigm without the need for coarse and fine reconstruction. We conduct several experiments to evaluate our method and demonstrate that it provides better upsampling.

关键词： Autonomous vehicle navigation deep learning for visual perception lidar transfer learning

来源：评论

学校读者我要写书评

暂无评论

learning Keypoints for Robotic Cloth Manipulation Using Synthetic Data

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第7期9卷 6528-6535页

作者： Lips, Thomas De Gusseme, Victor-Louis Wyffels, Francis Ghent Univ Imec AI & Robot Lab IDLab AIRO B-9052 Zwijnaarde Belgium

Assistive robots should be able to wash, fold or iron clothes. However, due to the variety, deformability and self-occlusions of clothes, creating robot systems for cloth manipulation is challenging. Synthetic data is a promising direction to improve generalization, but the sim-to-real gap limits its effectiveness. To advance the use of synthetic data for cloth manipulation tasks such as robotic folding, we present a synthetic data pipeline to train keypoint detectors for almost-flattened cloth items. To evaluate its performance, we have also collected a real-world dataset. We train detectors for both T-shirts, towels and shorts and obtain an average precision of 64% and an average keypoint distance of 18 pixels. Fine-tuning on real-world data improves performance to 74% mAP and an average distance of only 9 pixels. Furthermore, we describe failure modes of the keypoint detectors and compare different approaches to obtain cloth meshes and materials. We also quantify the remaining sim-to-real gap and argue that further improvements to the fidelity of cloth assets will be required to further reduce this gap. The code, dataset and trained models are available here.

关键词： Robots Synthetic data Pipelines Detectors Semantics Flexible printed circuits Deformation Data sets for robotic vision deep learning for visual perception simulation and animation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：