检索结果-内蒙古大学图书馆

Tube-NeRF: Efficient Imitation learning of Visuomotor Policies From MPC via Tube-Guided Data Augmentation and NeRFs

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第6期9卷 5544-5551页

作者： Tagliabue, Andrea How, Jonathan P. MIT Lab Informat & Decis Syst LIDS Cambridge MA 02139 USA

Imitation learning (IL) can train computationally-efficient sensorimotor policies from a resource-intensive model predictive controller (MPC), but it often requires many samples, leading to long training times or limited robustness. To address these issues, we combine IL with a variant of robust MPC that accounts for process and sensing uncertainties, and we design a data augmentation (DA) strategy that enables efficient learning of vision-based policies. The proposed DA method, named Tube-NeRF, leverages Neural Radiance Fields (NeRFs) to generate novel synthetic images, and uses properties of the robust MPC (the tube) to select relevant views and to efficiently compute the corresponding actions. We tailor our approach to the task of localization and trajectory tracking on a multirotor, by learning a visuomotor policy that generates control actions using images from the onboard camera as only source of horizontal position. Numerical evaluations show 80-fold increase in demonstration efficiency and a 50% reduction in training time over current IL methods. Additionally, our policies successfully transfer to a real multirotor, achieving low tracking errors despite large disturbances, with an onboard inference time of only 1.5 ms.

关键词： Imitation learning deep learning for visual perception aerial systems: perception and autonomy NeRF

来源：评论

学校读者我要写书评

暂无评论

Center Direction Network for Grasping Point Localization on Cloths

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第10期9卷 8913-8920页

作者： Tabernik, Domen Muhovic, Jon Urbas, Matej Skocaj, Danijel Univ Ljubljana Fac Comp & Informat Sci Ljubljana Slovenia Univ Ljubljana Fac Elect Engn Ljubljana 1000 Slovenia

Object grasping is a fundamental challenge in robotics and computer vision, critical for advancing robotic manipulation capabilities. Deformable objects, like fabrics and cloths, pose additional challenges due to their non-rigid nature. In this work, we introduce CeDiRNet-3DoF, a deep-learning model for grasp point detection, with a particular focus on cloth objects. CeDiRNet-3DoF employs center direction regression alongside a localization network, attaining first place in the perception task of ICRA 2023's Cloth Manipulation Challenge. Recognizing the lack of standardized benchmarks in the literature that hinder effective method comparison, we present the ViCoS Towel Dataset. This extensive benchmark dataset comprises 8,000 real and 12,000 synthetic images, serving as a robust resource for training and evaluating contemporary data-driven deep-learning approaches. Extensive evaluation revealed CeDiRNet-3DoF's robustness in real-world performance, outperforming state-of-the-art methods, including the latest transformer-based models. Our work bridges a crucial gap, offering a robust solution and benchmark for cloth grasping in computer vision and robotics.

关键词： Location awareness Benchmark testing Estimation Training Grasping Computer vision Decoding deep learning for visual perception data sets for robotic vision RGB-D perception

来源：评论

学校读者我要写书评

暂无评论

Implicit learning of Scene Geometry From Poses for Global Localization

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第2期9卷 955-962页

作者： Altillawi, Mohammad Li, Shile Prakhya, Sai Manoj Liu, Ziyuan Serrat, Joan Huawei Munich Res Ctr Robot & Simulat Team D-80992 Munich Germany Univ Autonoma Barcelona Comp Vis Ctr Dept Comp Sci Bellaterra 08193 Spain

Global visual localization estimates the absolute pose of a camera using a single image, in a previously mapped area. Obtaining the pose from a single image enables many robotics and augmented/virtual reality applications. Inspired by latest advances in deep learning, many existing approaches directly learn and regress 6 DoF pose from an input image. However, these methods do not fully utilize the underlying scene geometry for pose regression. The challenge in monocular relocalization is the minimal availability of supervised training data, which is just the corresponding 6 DoF poses of the images. In this letter, we propose to utilize these minimal available labels (i.e., poses) to learn the underlying 3D geometry of the scene and use the geometry to estimate the 6 DoF camera pose. We present a learning method that uses these pose labels and rigid alignment to learn two 3D geometric representations (X, Y, Z coordinates) of the scene, one in camera coordinate frame and the other in global coordinate frame. Given a single image, it estimates these two 3D scene representations, which are then aligned to estimate a pose that matches the pose label. This formulation allows for the active inclusion of additional learning constraints to minimize 3D alignment errors between the two 3D scene representations, and 2D re-projection errors between the 3D global scene representation and 2D image pixels, resulting in improved localization accuracy. During inference, our model estimates the 3D scene geometry in camera and global frames and aligns them rigidly to obtain pose in real-time. We evaluate our work on three common visual localization datasets, conduct ablation studies, and show that our method exceeds state-of-the-art regression methods' pose accuracy on all datasets.

关键词： Localization localization and mapping deep learning for visual perception visual learning

来源：评论

学校读者我要写书评

暂无评论

RoadRunner M&M - learning Multi-Range Multi-Resolution Traversability Maps for Autonomous Off-Road Navigation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第12期9卷 11425-11432页

作者： Patel, Manthan Frey, Jonas Atha, Deegan Spieler, Patrick Hutter, Marco Khattak, Shehryar Caltech Jet Prop Lab Pasadena CA 91125 USA Swiss Fed Inst Technol Robot Syst Lab CH-8092 Zurich Switzerland

Autonomous robot navigation in off-road environments requires a comprehensive understanding of the terrain geometry and traversability. The degraded perceptual conditions and sparse geometric information at longer ranges make the problem challenging especially when driving at high speeds. Furthermore, the sensing-to-mapping latency and the look-ahead map range can limit the maximum speed of the vehicle. Building on top of the recent work RoadRunner, in this work, we address the challenge of long-range ( +/- 100m ) traversability estimation. Our RoadRunner (M&M) is an end-to-end learning-based framework that directly predicts the traversability and elevation maps at multiple ranges ( +/- 50m , +/- 100m ) and resolutions ( 0.2m , 0.8m ) taking as input multiple images and a LiDAR voxel map. Our method is trained in a self-supervised manner by leveraging the dense supervision signal generated by fusing predictions from an existing traversability estimation stack (X-Racer) in hindsight and satellite Digital Elevation Maps. RoadRunner M&M achieves a significant improvement of up to 50% for elevation mapping and 30% for traversability estimation over RoadRunner, and is able to predict in 30% more regions compared to X-Racer while achieving real-time performance. Experiments on various out-of-distribution datasets also demonstrate that our data-driven approach starts to generalize to novel unstructured environments. We integrate our proposed framework in closed-loop with the path planner to demonstrate autonomous high-speed off-road robotic navigation in challenging real-world environments.

关键词： Laser radar Robot sensing systems Navigation Three-dimensional displays Image resolution Estimation Semantics Biological system modeling Point cloud compression visualization Field robots deep learning for visual perception mapping

来源：评论

学校读者我要写书评

暂无评论

Diver Identification Using Anthropometric DataRatios for Underwater Multi-Human-RobotCollaboration

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第4期9卷 3514-3521页

作者： Hong, Jungseok Enan, Sadman Sakib Sattar, Junaed Univ Minnesota Dept Comp Sci & Engn Minneapolis MN 55455 USA Univ Minnesota Minnesota Robot Inst Minneapolis MN 55455 USA

Recent advances in efficient design, perception algorithms, and computing hardware have made it possible to create improved human-robot interaction (HRI) capabilities for autonomous underwater vehicles (AUVs). To conduct secure missions as underwater human-robot teams, AUVs require the ability to accurately identify divers. However, this remains an open problem due to divers' challenging visual features, mainly caused by similar-looking scuba gear. In this letter, we present a novel algorithm that can perform diver identification using either pre-trained models or models trained during deployment. We exploit anthropometric data obtained from diver pose estimates to generate robust features that are invariant to changes in distance and photometric conditions. We also propose an embedding network that maximizes inter-class distances in the feature space and minimizes those for the intra-class features, which significantly improves classification performance. Furthermore, we present an end-to-end diver identification framework that operates on an AUV and evaluate the accuracy of the proposed algorithm. Quantitative results in controlled-water experiments show that our algorithm achieves a high level of accuracy in diver identification.

关键词： Pose estimation Robots Feature extraction Filtering Human-robot interaction Hip Faces Marine robotics human-robot collaboration deep learning for visual perception human detection and tracking

来源：评论

学校读者我要写书评

暂无评论

InfPose: Real-Time Infrared Multi-Human Pose Estimation for Edge Devices Based on Encoder-Decoder CNN Architecture

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第4期9卷 3672-3679页

作者： Xu, Xin Wei, Xinchao Xu, Yuelei Zhang, Zhaoxiang Gong, Kun Li, Huafeng Xiao, Leibing Northwestern Polytech Univ Unmanned Syst Res Inst Xian 710072 Peoples R China

Despite its remarkable performance, RGB-based Multi-human Pose Estimation (MPE) technology has many practical limitations, such as nighttime and smoggy environments. Infrared imaging is a valid substitution in these scenarios but needs an efficient and fast method for MPE. This letter aims to design an infrared MPE model based on the Encoder-Decoder CNN architecture, InfPose, which can perform real-time on edge devices. We first built a lightweight Encoder-Decoder CNN backbone based on hardware-friendly inverted residual blocks. Secondly, we utilized three methods to improve the capability of InfPose, including decoupling associative embedding head, multi-scale supervision, and cross-modal knowledge distillation. In addition, we gathered a wild infrared human pose dataset to train and evaluate our methods. Experiment results show that the proposed model is more robust and has less latency when inference on edge GPU platforms in comparison to the prevailing mainstream models. The inference time for InfPose on Xavier NX was recorded as 27.7 ms (approximate to 37 fps) and maintained sufficient accuracy for use. This research can be applied to human-machine interaction of autonomous vehicles or intelligent robots in nighttime or other scenes with poor visibility.

关键词： Pose estimation Decoding Convolution Real-time systems Performance evaluation Image resolution Head deep learning for visual perception gesture posture and facial expressions multi-human pose estimation infrared vision

来源：评论

学校读者我要写书评

暂无评论

Multi-Camera Unified Pre-Training via 3D Scene Reconstruction

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第4期9卷 3243-3250页

作者： Min, Chen Xiao, Liang Zhao, Dawei Nie, Yiming Dai, Bin Peking Univ Sch Comp Sci Beijing 100871 Peoples R China Def Innovat Inst Unmanned Syst Technol Res Ctr Beijing 10073 Peoples R China Intelligent Game & Decis Lab Beijing 100071 Peoples R China

Multi-camera 3D perception has emerged as a prominent research field in autonomous driving, offering a viable and cost-effective alternative to LiDAR-based solutions. The existing multi-camera algorithms primarily rely on monocular 2D pre-training. However, the monocular 2D pre-training overlooks the spatial and temporal correlations among the multi-camera system. To address this limitation, we propose the first multi-camera unified pre-training framework, called UniScene, which involves initially reconstructing the 3D scene as the foundational stage and subsequently fine-tuning the model on downstream tasks. Specifically, we employ Occupancy as the general representation for the 3D scene, enabling the model to grasp geometric priors of the surrounding world through pre-training. A significant benefit of UniScene is its capability to utilize a considerable volume of unlabeled image-LiDAR pairs for pre-training purposes. The proposed multi-camera unified pre-training framework demonstrates promising results in key tasks such as multi-camera 3D object detection and surrounding semantic scene completion. When compared to monocular pre-training methods, UniScene shows a significant improvement of about 2.0% in mAP and 2.0% in NDS for multi-camera 3D object detection, as well as a 3% increase in mIoU for surrounding semantic scene completion. By adopting our unified pre-training method, a 25% reduction in 3D training annotation costs can be achieved, offering significant practical value for the implementation of real-world autonomous driving.

关键词： Autonomous vehicles Decoding Laser radar Autonomous vehicle navigation deep learning for visual perception recognition representation learning

来源：评论

学校读者我要写书评

暂无评论

High-Speed Detector for Low-Powered Devices in Aerial Grasping

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第5期9卷 4623-4630页

作者： Kumar, Ashish Behera, Laxmidhar Indian Inst Technol IIT EE Kanpur 208016 India

Autonomous aerial harvesting is a highly complex problem because it requires numerous interdisciplinary algorithms to be executed on mini low-powered computing devices. Object detection is one such algorithm that is compute-hungry. In this context, we make the following contributions: (i) Fast Fruit Detector (FFD), a resource-efficient, single-stage, and postprocessing-free object detector based on our novel latent object representation (LOR) module, query assignment, and prediction strategy. FFD achieves 100 FPS @ FP 32 precision on the latest 10 W NVIDIA Jetson-NX embedded device while co-existing with other time-critical sub-systems such as control, grasping, SLAM, a major achievement of this work, (ii) a method to generate vast amounts of training data without exhaustive manual labelling of fruit images since they consist of a large number of instances, which increases the labelling cost and time, and (iii) an open-source fruit detection dataset having plenty of very small-sized instances that are difficult to detect. Our exhaustive evaluations on our and MinneApple dataset show that FFD, being only a single-scale detector, is more accurate than many representative detectors, e.g. FFD is better than single-scale Faster-RCNN by 10.7 AP, multi-scale Faster-RCNN by 2.3 AP, and better than latest single-scale YOLO-v 8 by 8 AP and multi-scale YOLO-v 8 by 0.3 while being considerably faster.

关键词： Aerial systems: applications agricultural automation deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

State Estimation and Belief Space Planning Under Epistemic Uncertainty for learning-Based perception Systems

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第6期9卷 5118-5125页

作者： Nagami, Keiko Schwager, Mac Stanford Univ Dept Aeronaut & Astronaut Stanford CA 94305 USA

learning-based models for robot perception are known to suffer from two distinct sources of error: aleatoric and epistemic. Aleatoric uncertainty arises from inherently noisy training data and is easily quantified from residual errors during training. Conversely, epistemic uncertainty arises from a lack of training data, appearing in out-of-distribution operating regimes, and is difficult to quantify. Most existing state estimation methods handle aleatoric uncertainty through a learned noise model, but ignore epistemic uncertainty. In this work, we propose: (i) an epistemic Kalman filter (EpiKF) to incorporate epistemic uncertainty into state estimation with learned perception models, and (ii) an epistemic belief space planner (EpiBSP) that builds on the EpiKF to plan trajectories to avoid areas of high epistemic and aleatoric uncertainty. Our key insight is to train a generative model that predicts measurements from states, "inverting" the learned perception model that predicts states from measurements. We compose these two models in a sampling scheme to give a well-calibrated online estimate of combined epistemic and aleatoric uncertainty. We demonstrate our method in a vision-based drone racing scenario, and show superior performance to existing methods that treat measurement noise covariance as a learned output of the perception model.

关键词： Uncertainty Planning Measurement uncertainty Trajectory Neural networks Training data Training Aerial systems: perception and autonomy planning under uncertainty deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

AnyLoc: Towards Universal visual Place Recognition

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第2期9卷 1286-1293页

作者： Keetha, Nikhil Mishra, Avneesh Karhade, Jay Jatavallabhula, Krishna Murthy Scherer, Sebastian Krishna, Madhava Garg, Sourav Carnegie Mellon Univ Robot Inst Pittsburgh PA 15213 USA Int Inst Informat Technol Hyderabad 500032 India MIT Cambridge MA 02139 USA Univ Adelaide Australian Inst Machine Learning Adelaide SA 5000 Australia

visual Place Recognition (VPR) is vital for robot localization. To date, the most performant VPR approaches are environment- and task-specific: while they exhibit strong performance in structured environments (predominantly urban driving), their performance degrades severely in unstructured environments, rendering most approaches brittle to robust real-world deployment. In this work, we develop a universal solution to VPR - a technique that works across a broad range of structured and unstructured environments (urban, outdoors, indoors, aerial, underwater, and subterranean environments) without any re-training or finetuning. We demonstrate that general-purpose feature representations derived from off-the-shelf self-supervised models with no VPR-specific training are the right substrate upon which to build such a universal VPR solution. Combining these derived features with unsupervised feature aggregation enables our suite of methods, AnyLoc, to achieve up to 4x significantly higher performance than existing approaches. We further obtain a 6% improvement in performance by characterizing the semantic properties of these features, uncovering unique domains which encapsulate datasets from similar environments. Our detailed experiments and analysis lay a foundation for building VPR solutions that may be deployed anywhere, anytime, and across anyview.

关键词： Feature extraction Training visualization Task analysis Vocabulary Semantics Robustness Localization recognition deep learning for visual perception vision-based navigation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：