检索结果-内蒙古大学图书馆

Exploiting Depth Priors for Few-Shot Neural Radiance Field Reconstruction

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第11期9卷 9844-9851页

作者： Chen, Shuya Li, Zheyang Zhu, Hao Tan, Wenming Ren, Ye Xiang, Zhiyu Hikvis Res Inst Hangzhou 310000 Peoples R China Zhejiang Univ Coll Informat Sci & Elect Engn Hangzhou 310000 Peoples R China Zhejiang Univ Zhejiang Prov Key Lab Informat Proc Commun & Netwo Hangzhou 310000 Peoples R China

The performance of neural radiance field technologies deteriorates rapidly when sparse views are used as input. In this paper, we propose a simulated viewpoint enhancement for surface reconstruction that extracts diverse geometric features from the depth to address this limitation. We design a novel pseudo-view generation approach to simulate dense viewpoints, in which the RGB-D pairs are converted into new images corresponding to the translational poses through reprojection. To address potential holes in the pseudo-views resulting from occlusion in the reprojecting process, we propose a hole handling mechanism to exclude them during training. Additionally, normal information is derived from the depth, serving as supervision to enhance the scene geometry. Experiments conducted on various baseline models and datasets demonstrate that our algorithm effectively utilizes the inherent geometry embedded in the depth maps, leading to a significant improvement in the quality of 3D surface reconstruction compared to existing methods.

关键词： Image reconstruction Point cloud compression Neural radiance field Surface reconstruction Training Image color analysis Rendering (computer graphics) deep learning for visual perception computer vision for automation surface reconstruction neural radiance field RGB-D

来源：评论

学校读者我要写书评

暂无评论

KVN: Keypoints Voting Network With Differentiable RANSAC for Stereo Pose Estimation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第4期9卷 3498-3505页

作者： Donadi, Ivano Pretto, Alberto Univ Padua Dept Informat Engn I-35131 Padua Italy

Object pose estimation is a fundamental computer vision task exploited in several robotics and augmented reality applications. Many established approaches rely on predicting 2D-3D keypoint correspondences using RANSAC (Random sample consensus) and estimating the object pose using the PnP (Perspective-n-Point) algorithm. Being RANSAC non-differentiable, correspondences cannot be directly learned in an end-to-end fashion. In this letter, we address the stereo image-based object pose estimation problem by i) introducing a differentiable RANSAC layer into a well-known monocular pose estimation network;ii) exploiting an uncertainty-driven multi-view PnP solver which can fuse information from multiple views. We evaluate our approach on a challenging public stereo object pose estimation dataset and a custom-built dataset we call Transparent Tableware Dataset (TTD), yielding state-of-the-art results against other recent approaches. Furthermore, in our ablation study, we show that the differentiable RANSAC layer plays a significant role in the accuracy of the proposed method. We release with this letter the code of our method and the TTD dataset.

关键词： Computer vision for automation deep learning for visual perception perception for grasping and manipulation

来源：评论

学校读者我要写书评

暂无评论

Mono-Camera-Only Target Chasing for a Drone in a Dense Environment by Cross-Modal learning

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第8期9卷 7254-7261页

作者： Yoo, Seungyeon Jung, Seungwoo Lee, Yunwoo Shim, Dongseok Kim, H. Jin Seoul Natl Univ Automat & Syst Res Inst Dept Aerosp Engn Seoul 08826 South Korea Seoul Natl Univ Artificial Intelligence Inst SNU Seoul 08826 South Korea Seoul Natl Univ Interdisciplinary Program AI Seoul 08826 South Korea

Chasing a dynamic target in a dense environment is one of the challenging applications of autonomous drones. The task requires multi-modal data, such as RGB and depth, to accomplish safe and robust maneuver. However, using different types of modalities can be difficult due to the limited capacity of drones in aspects of hardware complexity and sensor cost. Our framework resolves such restrictions in the target chasing task by using only a monocular camera instead of multiple sensor inputs. From an RGB input, the perception module can extract a cross-modal representation containing information from multiple data modalities. To learn cross-modal representations at training time, we employ variational autoencoder (VAE) structures and the joint objective function across heterogeneous data. Subsequently, using latent vectors acquired from the pre-trained perception module, the planning module generates a proper next-time-step waypoint by imitation learning of the expert, which performs a numerical optimization using the privileged RGB-D data. Furthermore, the planning module considers temporal information of the target to improve tracking performance through consecutive cross-modal representations. Ultimately, we demonstrate the effectiveness of our framework through the reconstruction results of the perception module, the target chasing performance of the planning module, and the zero-shot sim-to-real deployment of a drone.

关键词： Drones Task analysis Planning Target tracking Vehicle dynamics Training Image reconstruction Vision-based navigation visual learning deep learning for visual perception deep learning methods

来源：评论

学校读者我要写书评

暂无评论

Bathymetric Surveying With Imaging Sonar Using Neural Volume Rendering

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第9期9卷 8146-8153页

作者： Xie, Yiping Troni, Giancarlo Bore, Nils Folkesson, John KTH Royal Inst Technol Robot Percept & Learning Div S-10044 Stockholm Sweden Monterey Bay Aquarium Res Inst Moss Landing CA 95039 USA Ocean Infin SE-42671 Vastra Frolunda Sweden

This research addresses the challenge of estimating bathymetry from imaging sonars where the state-of-the-art works have primarily relied on either supervised learning with ground-truth labels or surface rendering based on the Lambertian assumption. In this letter, we propose a novel, self-supervised framework based on volume rendering for reconstructing bathymetry using forward-looking sonar (FLS) data collected during standard surveys. We represent the seafloor as a neural heightmap encapsulated with a parametric multi-resolution hash encoding scheme and model the sonar measurements with a differentiable renderer using sonar volumetric rendering employed with hierarchical sampling techniques. Additionally, we model the horizontal and vertical beam patterns and estimate them jointly with the bathymetry. We evaluate the proposed method quantitatively on simulation and field data collected by remotely operated vehicles (ROVs) during low-altitude surveys. Results show that the proposed method outperforms the current state-of-the-art approaches that use imaging sonars for seabed mapping. We also demonstrate that the proposed approach can potentially be used to increase the resolution of a low-resolution prior map with FLS data from low-altitude surveys.

关键词： Sonar Bathymetry Image reconstruction Three-dimensional displays Rendering (computer graphics) Surveys Encoding Bathymetric reconstruction deep learning methods deep learning for visual perception mapping marine robotics

来源：评论

学校读者我要写书评

暂无评论

learning Keypoints for Robotic Cloth Manipulation Using Synthetic Data

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第7期9卷 6528-6535页

作者： Lips, Thomas De Gusseme, Victor-Louis Wyffels, Francis Ghent Univ Imec AI & Robot Lab IDLab AIRO B-9052 Zwijnaarde Belgium

Assistive robots should be able to wash, fold or iron clothes. However, due to the variety, deformability and self-occlusions of clothes, creating robot systems for cloth manipulation is challenging. Synthetic data is a promising direction to improve generalization, but the sim-to-real gap limits its effectiveness. To advance the use of synthetic data for cloth manipulation tasks such as robotic folding, we present a synthetic data pipeline to train keypoint detectors for almost-flattened cloth items. To evaluate its performance, we have also collected a real-world dataset. We train detectors for both T-shirts, towels and shorts and obtain an average precision of 64% and an average keypoint distance of 18 pixels. Fine-tuning on real-world data improves performance to 74% mAP and an average distance of only 9 pixels. Furthermore, we describe failure modes of the keypoint detectors and compare different approaches to obtain cloth meshes and materials. We also quantify the remaining sim-to-real gap and argue that further improvements to the fidelity of cloth assets will be required to further reduce this gap. The code, dataset and trained models are available here.

关键词： Robots Synthetic data Pipelines Detectors Semantics Flexible printed circuits Deformation Data sets for robotic vision deep learning for visual perception simulation and animation

来源：评论

学校读者我要写书评

暂无评论

ShaSTA: Modeling Shape and Spatio-Temporal Affinities for 3D Multi-Object Tracking

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第5期9卷 4273-4280页

作者： Sadjadpour, Tara Li, Jie Ambrus, Rares Bohg, Jeannette Stanford Univ Sch Engn Comp Sci Dept Stanford CA 94305 USA NVIDIA Santa Clara CA 95051 USA Toyota Res Inst Los Altos CA 94022 USA

Multi-object tracking (MOT) is a cornerstone capability of any robotic system. Tracking quality is largely dependent on the quality of input detections. In many applications, such as autonomous driving, it is preferable to over-detect objects to avoid catastrophic outcomes due to missed detections. As a result, current state-of-the-art 3D detectors produce high rates of false-positives to ensure a low number of false-negatives. This can negatively affect tracking by making data association and track lifecycle management more challenging. Additionally, occasional false-negative detections due to difficult scenarios like occlusions can harm tracking performance. To address these issues in a unified framework, we propose ShaSTA which learns shape and spatio-temporal affinities between tracks and detections in consecutive frames. The affinity is a probabilistic matching that leads to robust data association, track lifecycle management, false-positive elimination, false-negative propagation, and sequential track confidence refinement. We offer the first self-contained framework that addresses all aspects of the 3D MOT problem. We quantitatively evaluate ShaSTA on the nuScenes tracking benchmark with 5 metrics, including the most common tracking accuracy metric called AMOTA, to demonstrate how ShaSTA may impact the ultimate goal of an autonomous mobile agent. ShaSTA achieves 1st place amongst LiDAR-only trackers that use CenterPoint detections.

关键词： Computer vision for transportation deep learning for visual perception visual tracking

来源：评论

学校读者我要写书评

暂无评论

Rapid-Mapping: LiDAR-visual Implicit Neural Representations for Real-Time Dense Mapping

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第9期9卷 8154-8161页

作者： Zhang, Hanwen Zou, Yujie Yan, Zhewen Cheng, Hui Sun Yat Sen Univ Sch Comp Sci & Engn Guangzhou 510006 Peoples R China

Real-time dense mapping with high-fidelity textures in large-scale environments is such a challenge in robots, digital twins, and AR/VR applications. Neural Radiance Field (NeRF) has demonstrated remarkable capabilities in capturing intricate details and saving memory space, which provides significant advantages in the fine-grained reconstruction of large-scale scenes. Existing LiDAR-based mapping methods have not harnessed NeRF's ability to capture textures. In this letter, we propose the first real-time LiDAR-visual mapping method in large-scale indoor and outdoor environments, named Rapid-Mapping, that utilizes implicit neural representations and preserves high-fidelity textures. First of all, to align the camera image and LiDAR depth, we propose a method for extrinsic refinement to mitigate the issue of texture blurring caused by long-range and extended temporal scale measurements. Also, to alleviate the effects of lighting conditions and camera hardware interference, we utilize prior hue information to constrain the inverse affine transformation. Extensive experiments validate that Rapid-Mapping enables real-time dense mapping in large-scale complex indoor and outdoor scenes, exhibiting more detailed realistic textures and more accurate geometry compared to existing methods.

关键词： deep learning for visual perception mapping visual learning deep learning for visual perception mapping visual learning

来源：评论

学校读者我要写书评

暂无评论

MotionPerceiver: Real-Time Occupancy Forecasting for Embedded Systems

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第3期9卷 2822-2829页

作者： Ferenczi, Bryce Burke, Michael Drummond, Tom Monash Univ Dept Elect & Comp Syst Engn Clayton Vic 3178 Australia Univ Melbourne Dept Comp & Informat Syst Parkville Vic 3010 Australia

This work introduces a novel and adaptable architecture designed for real-time occupancy forecasting that outperforms existing state-of-the-art models on the Waymo Open Motion Dataset in Soft IOU. The proposed model uses recursive latent state estimation with learned transformer-based functions to effectively update and evolve the state. This enables highly efficient real-time inference on embedded systems, as profiled on an Nvidia Xavier AGX. Our model, MotionPerceiver, achieves this by encoding a scene into a latent state that evolves in time through self-attention mechanisms. Additionally, it incorporates relevant scene observations, such as traffic signals, road topology and agent detections, through cross-attention mechanisms. This forms an efficient data-streaming architecture, that contrasts with the expensive, fixed-sequence input common in existing models. The architecture also offers the distinct advantage of generating occupancy predictions through localized querying based on a point-of-interest, as opposed to generating fixed-size occupancy images that render potentially irrelevant regions.

关键词： Computer vision for transportation deep learning for visual perception representation learning

来源：评论

学校读者我要写书评

暂无评论

RomniStereo: Recurrent Omnidirectional Stereo Matching

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第3期9卷 2511-2518页

作者： Jiang, Hualie Xu, Rui Tan, Minglang Jiang, Wenjie Insta360 Res Shenzhen 518172 Peoples R China

Omnidirectional stereo matching (OSM) is an essential and reliable means for $360{\circ }$ depth sensing. However, following earlier works on conventional stereo matching, prior state-of-the-art (SOTA) methods rely on... 详细信息

Omnidirectional stereo matching (OSM) is an essential and reliable means for $360<^>{\circ }$ depth sensing. However, following earlier works on conventional stereo matching, prior state-of-the-art (SOTA) methods rely on a 3D encoder-decoder block to regularize the cost volume, causing the whole system complicated and sub-optimal results. Recently, the Recurrent All-pairs Field Transforms (RAFT) based approach employs the recurrent update in 2D and has efficiently improved image-matching tasks, i.e., optical flow, and stereo matching. To bridge the gap between OSM and RAFT, we mainly propose an opposite adaptive weighting scheme to seamlessly transform the outputs of spherical sweeping of OSM into the required inputs for the recurrent update, thus creating a recurrent omnidirectional stereo matching (RomniStereo) algorithm. Furthermore, we introduce two techniques, i.e., grid embedding and adaptive context feature generation, which also contribute to RomniStereo's performance. Our best model improves the average MAE metric by 40.7% over the previous SOTA baseline across five datasets. When visualizing the results, our models demonstrate clear advantages on both synthetic and realistic examples.

关键词： Cameras Feature extraction Three-dimensional displays Costs Correlation Estimation Robot vision systems deep learning for visual perception omnidirectional vision stereo matching depth estimation

来源：评论

学校读者我要写书评

暂无评论

Clio: Real-Time Task-Driven Open-Set 3D Scene Graphs

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第10期9卷 8921-8928页

作者： Maggio, Dominic Chang, Yun Hughes, Nathan Trang, Matthew Griffith, Dan Dougherty, Carlyn Cristofalo, Eric Schmid, Lukas Carlone, Luca MIT Lab Informat & Decis Syst Cambridge MA 02139 USA Lincoln Lab MIT Lexington MA 02421 USA

Modern tools for class-agnostic image segmentation (e.g., SegmentAnything) and open-set semantic understanding (e.g., CLIP) provide unprecedented opportunities for robot perception and mapping. While traditional closed-set metric-semantic maps were restricted to tens or hundreds of semantic classes, we can now build maps with a plethora of objects and countless semantic variations. This leaves us with a fundamental question: what is the right granularity for the objects (and, more generally, for the semantic concepts) the robot has to include in its map representation? While related work implicitly chooses a level of granularity by tuning thresholds for object detection, we argue that such a choice is intrinsically task-dependent. The first contribution of this paper is to propose a task-driven 3D scene understanding problem, where the robot is given a list of tasks in natural language, and has to select the granularity and the subset of objects and scene structure to retain in its map that is sufficient to complete the tasks. We show that this problem can be naturally formulated using the Information Bottleneck (IB), an established information-theoretic framework to discuss task-relevance. The second contribution is an algorithm for task-driven 3D scene understanding based on an Agglomerative IB approach, that is able to cluster 3D primitives in the environment into task-relevant objects and regions. The third contribution is to integrate our task-driven clustering algorithm into a real-time pipeline, named Clio, that constructs a hierarchical 3D scene graph of the environment online and using only onboard compute. Our final contribution is an extensive experimental campaign showing that Clio not only allows real-time construction of compact open-set 3D scene graphs, but also improves the accuracy of task execution by limiting the map to relevant semantic concepts.

关键词： Task analysis Three-dimensional displays Semantics Robots Real-time systems Image segmentation Natural languages Mapping deep learning for visual perception semantic scene understanding

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：