检索结果-内蒙古大学图书馆

MotionPerceiver: Real-Time Occupancy Forecasting for Embedded Systems

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第3期9卷 2822-2829页

作者： Ferenczi, Bryce Burke, Michael Drummond, Tom Monash Univ Dept Elect & Comp Syst Engn Clayton Vic 3178 Australia Univ Melbourne Dept Comp & Informat Syst Parkville Vic 3010 Australia

This work introduces a novel and adaptable architecture designed for real-time occupancy forecasting that outperforms existing state-of-the-art models on the Waymo Open Motion Dataset in Soft IOU. The proposed model uses recursive latent state estimation with learned transformer-based functions to effectively update and evolve the state. This enables highly efficient real-time inference on embedded systems, as profiled on an Nvidia Xavier AGX. Our model, MotionPerceiver, achieves this by encoding a scene into a latent state that evolves in time through self-attention mechanisms. Additionally, it incorporates relevant scene observations, such as traffic signals, road topology and agent detections, through cross-attention mechanisms. This forms an efficient data-streaming architecture, that contrasts with the expensive, fixed-sequence input common in existing models. The architecture also offers the distinct advantage of generating occupancy predictions through localized querying based on a point-of-interest, as opposed to generating fixed-size occupancy images that render potentially irrelevant regions.

关键词： Computer vision for transportation deep learning for visual perception representation learning

来源：评论

学校读者我要写书评

暂无评论

ShaSTA: Modeling Shape and Spatio-Temporal Affinities for 3D Multi-Object Tracking

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第5期9卷 4273-4280页

作者： Sadjadpour, Tara Li, Jie Ambrus, Rares Bohg, Jeannette Stanford Univ Sch Engn Comp Sci Dept Stanford CA 94305 USA NVIDIA Santa Clara CA 95051 USA Toyota Res Inst Los Altos CA 94022 USA

Multi-object tracking (MOT) is a cornerstone capability of any robotic system. Tracking quality is largely dependent on the quality of input detections. In many applications, such as autonomous driving, it is preferable to over-detect objects to avoid catastrophic outcomes due to missed detections. As a result, current state-of-the-art 3D detectors produce high rates of false-positives to ensure a low number of false-negatives. This can negatively affect tracking by making data association and track lifecycle management more challenging. Additionally, occasional false-negative detections due to difficult scenarios like occlusions can harm tracking performance. To address these issues in a unified framework, we propose ShaSTA which learns shape and spatio-temporal affinities between tracks and detections in consecutive frames. The affinity is a probabilistic matching that leads to robust data association, track lifecycle management, false-positive elimination, false-negative propagation, and sequential track confidence refinement. We offer the first self-contained framework that addresses all aspects of the 3D MOT problem. We quantitatively evaluate ShaSTA on the nuScenes tracking benchmark with 5 metrics, including the most common tracking accuracy metric called AMOTA, to demonstrate how ShaSTA may impact the ultimate goal of an autonomous mobile agent. ShaSTA achieves 1st place amongst LiDAR-only trackers that use CenterPoint detections.

关键词： Computer vision for transportation deep learning for visual perception visual tracking

来源：评论

学校读者我要写书评

暂无评论

Clio: Real-Time Task-Driven Open-Set 3D Scene Graphs

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第10期9卷 8921-8928页

作者： Maggio, Dominic Chang, Yun Hughes, Nathan Trang, Matthew Griffith, Dan Dougherty, Carlyn Cristofalo, Eric Schmid, Lukas Carlone, Luca MIT Lab Informat & Decis Syst Cambridge MA 02139 USA Lincoln Lab MIT Lexington MA 02421 USA

Modern tools for class-agnostic image segmentation (e.g., SegmentAnything) and open-set semantic understanding (e.g., CLIP) provide unprecedented opportunities for robot perception and mapping. While traditional closed-set metric-semantic maps were restricted to tens or hundreds of semantic classes, we can now build maps with a plethora of objects and countless semantic variations. This leaves us with a fundamental question: what is the right granularity for the objects (and, more generally, for the semantic concepts) the robot has to include in its map representation? While related work implicitly chooses a level of granularity by tuning thresholds for object detection, we argue that such a choice is intrinsically task-dependent. The first contribution of this paper is to propose a task-driven 3D scene understanding problem, where the robot is given a list of tasks in natural language, and has to select the granularity and the subset of objects and scene structure to retain in its map that is sufficient to complete the tasks. We show that this problem can be naturally formulated using the Information Bottleneck (IB), an established information-theoretic framework to discuss task-relevance. The second contribution is an algorithm for task-driven 3D scene understanding based on an Agglomerative IB approach, that is able to cluster 3D primitives in the environment into task-relevant objects and regions. The third contribution is to integrate our task-driven clustering algorithm into a real-time pipeline, named Clio, that constructs a hierarchical 3D scene graph of the environment online and using only onboard compute. Our final contribution is an extensive experimental campaign showing that Clio not only allows real-time construction of compact open-set 3D scene graphs, but also improves the accuracy of task execution by limiting the map to relevant semantic concepts.

关键词： Task analysis Three-dimensional displays Semantics Robots Real-time systems Image segmentation Natural languages Mapping deep learning for visual perception semantic scene understanding

来源：评论

学校读者我要写书评

暂无评论

Tube-NeRF: Efficient Imitation learning of Visuomotor Policies From MPC via Tube-Guided Data Augmentation and NeRFs

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第6期9卷 5544-5551页

作者： Tagliabue, Andrea How, Jonathan P. MIT Lab Informat & Decis Syst LIDS Cambridge MA 02139 USA

Imitation learning (IL) can train computationally-efficient sensorimotor policies from a resource-intensive model predictive controller (MPC), but it often requires many samples, leading to long training times or limited robustness. To address these issues, we combine IL with a variant of robust MPC that accounts for process and sensing uncertainties, and we design a data augmentation (DA) strategy that enables efficient learning of vision-based policies. The proposed DA method, named Tube-NeRF, leverages Neural Radiance Fields (NeRFs) to generate novel synthetic images, and uses properties of the robust MPC (the tube) to select relevant views and to efficiently compute the corresponding actions. We tailor our approach to the task of localization and trajectory tracking on a multirotor, by learning a visuomotor policy that generates control actions using images from the onboard camera as only source of horizontal position. Numerical evaluations show 80-fold increase in demonstration efficiency and a 50% reduction in training time over current IL methods. Additionally, our policies successfully transfer to a real multirotor, achieving low tracking errors despite large disturbances, with an onboard inference time of only 1.5 ms.

关键词： Imitation learning deep learning for visual perception aerial systems: perception and autonomy NeRF

来源：评论

学校读者我要写书评

暂无评论

Implicit learning of Scene Geometry From Poses for Global Localization

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第2期9卷 955-962页

作者： Altillawi, Mohammad Li, Shile Prakhya, Sai Manoj Liu, Ziyuan Serrat, Joan Huawei Munich Res Ctr Robot & Simulat Team D-80992 Munich Germany Univ Autonoma Barcelona Comp Vis Ctr Dept Comp Sci Bellaterra 08193 Spain

Global visual localization estimates the absolute pose of a camera using a single image, in a previously mapped area. Obtaining the pose from a single image enables many robotics and augmented/virtual reality applications. Inspired by latest advances in deep learning, many existing approaches directly learn and regress 6 DoF pose from an input image. However, these methods do not fully utilize the underlying scene geometry for pose regression. The challenge in monocular relocalization is the minimal availability of supervised training data, which is just the corresponding 6 DoF poses of the images. In this letter, we propose to utilize these minimal available labels (i.e., poses) to learn the underlying 3D geometry of the scene and use the geometry to estimate the 6 DoF camera pose. We present a learning method that uses these pose labels and rigid alignment to learn two 3D geometric representations (X, Y, Z coordinates) of the scene, one in camera coordinate frame and the other in global coordinate frame. Given a single image, it estimates these two 3D scene representations, which are then aligned to estimate a pose that matches the pose label. This formulation allows for the active inclusion of additional learning constraints to minimize 3D alignment errors between the two 3D scene representations, and 2D re-projection errors between the 3D global scene representation and 2D image pixels, resulting in improved localization accuracy. During inference, our model estimates the 3D scene geometry in camera and global frames and aligns them rigidly to obtain pose in real-time. We evaluate our work on three common visual localization datasets, conduct ablation studies, and show that our method exceeds state-of-the-art regression methods' pose accuracy on all datasets.

关键词： Localization localization and mapping deep learning for visual perception visual learning

来源：评论

学校读者我要写书评

暂无评论

Center Direction Network for Grasping Point Localization on Cloths

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第10期9卷 8913-8920页

作者： Tabernik, Domen Muhovic, Jon Urbas, Matej Skocaj, Danijel Univ Ljubljana Fac Comp & Informat Sci Ljubljana Slovenia Univ Ljubljana Fac Elect Engn Ljubljana 1000 Slovenia

Object grasping is a fundamental challenge in robotics and computer vision, critical for advancing robotic manipulation capabilities. Deformable objects, like fabrics and cloths, pose additional challenges due to their non-rigid nature. In this work, we introduce CeDiRNet-3DoF, a deep-learning model for grasp point detection, with a particular focus on cloth objects. CeDiRNet-3DoF employs center direction regression alongside a localization network, attaining first place in the perception task of ICRA 2023's Cloth Manipulation Challenge. Recognizing the lack of standardized benchmarks in the literature that hinder effective method comparison, we present the ViCoS Towel Dataset. This extensive benchmark dataset comprises 8,000 real and 12,000 synthetic images, serving as a robust resource for training and evaluating contemporary data-driven deep-learning approaches. Extensive evaluation revealed CeDiRNet-3DoF's robustness in real-world performance, outperforming state-of-the-art methods, including the latest transformer-based models. Our work bridges a crucial gap, offering a robust solution and benchmark for cloth grasping in computer vision and robotics.

关键词： Location awareness Benchmark testing Estimation Training Grasping Computer vision Decoding deep learning for visual perception data sets for robotic vision RGB-D perception

来源：评论

学校读者我要写书评

暂无评论

RoadRunner M&M - learning Multi-Range Multi-Resolution Traversability Maps for Autonomous Off-Road Navigation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第12期9卷 11425-11432页

作者： Patel, Manthan Frey, Jonas Atha, Deegan Spieler, Patrick Hutter, Marco Khattak, Shehryar Caltech Jet Prop Lab Pasadena CA 91125 USA Swiss Fed Inst Technol Robot Syst Lab CH-8092 Zurich Switzerland

Autonomous robot navigation in off-road environments requires a comprehensive understanding of the terrain geometry and traversability. The degraded perceptual conditions and sparse geometric information at longer ranges make the problem challenging especially when driving at high speeds. Furthermore, the sensing-to-mapping latency and the look-ahead map range can limit the maximum speed of the vehicle. Building on top of the recent work RoadRunner, in this work, we address the challenge of long-range ( +/- 100m ) traversability estimation. Our RoadRunner (M&M) is an end-to-end learning-based framework that directly predicts the traversability and elevation maps at multiple ranges ( +/- 50m , +/- 100m ) and resolutions ( 0.2m , 0.8m ) taking as input multiple images and a LiDAR voxel map. Our method is trained in a self-supervised manner by leveraging the dense supervision signal generated by fusing predictions from an existing traversability estimation stack (X-Racer) in hindsight and satellite Digital Elevation Maps. RoadRunner M&M achieves a significant improvement of up to 50% for elevation mapping and 30% for traversability estimation over RoadRunner, and is able to predict in 30% more regions compared to X-Racer while achieving real-time performance. Experiments on various out-of-distribution datasets also demonstrate that our data-driven approach starts to generalize to novel unstructured environments. We integrate our proposed framework in closed-loop with the path planner to demonstrate autonomous high-speed off-road robotic navigation in challenging real-world environments.

关键词： Laser radar Robot sensing systems Navigation Three-dimensional displays Image resolution Estimation Semantics Biological system modeling Point cloud compression visualization Field robots deep learning for visual perception mapping

来源：评论

学校读者我要写书评

暂无评论

Diver Identification Using Anthropometric DataRatios for Underwater Multi-Human-RobotCollaboration

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第4期9卷 3514-3521页

作者： Hong, Jungseok Enan, Sadman Sakib Sattar, Junaed Univ Minnesota Dept Comp Sci & Engn Minneapolis MN 55455 USA Univ Minnesota Minnesota Robot Inst Minneapolis MN 55455 USA

Recent advances in efficient design, perception algorithms, and computing hardware have made it possible to create improved human-robot interaction (HRI) capabilities for autonomous underwater vehicles (AUVs). To conduct secure missions as underwater human-robot teams, AUVs require the ability to accurately identify divers. However, this remains an open problem due to divers' challenging visual features, mainly caused by similar-looking scuba gear. In this letter, we present a novel algorithm that can perform diver identification using either pre-trained models or models trained during deployment. We exploit anthropometric data obtained from diver pose estimates to generate robust features that are invariant to changes in distance and photometric conditions. We also propose an embedding network that maximizes inter-class distances in the feature space and minimizes those for the intra-class features, which significantly improves classification performance. Furthermore, we present an end-to-end diver identification framework that operates on an AUV and evaluate the accuracy of the proposed algorithm. Quantitative results in controlled-water experiments show that our algorithm achieves a high level of accuracy in diver identification.

关键词： Pose estimation Robots Feature extraction Filtering Human-robot interaction Hip Faces Marine robotics human-robot collaboration deep learning for visual perception human detection and tracking

来源：评论

学校读者我要写书评

暂无评论

InfPose: Real-Time Infrared Multi-Human Pose Estimation for Edge Devices Based on Encoder-Decoder CNN Architecture

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第4期9卷 3672-3679页

作者： Xu, Xin Wei, Xinchao Xu, Yuelei Zhang, Zhaoxiang Gong, Kun Li, Huafeng Xiao, Leibing Northwestern Polytech Univ Unmanned Syst Res Inst Xian 710072 Peoples R China

Despite its remarkable performance, RGB-based Multi-human Pose Estimation (MPE) technology has many practical limitations, such as nighttime and smoggy environments. Infrared imaging is a valid substitution in these scenarios but needs an efficient and fast method for MPE. This letter aims to design an infrared MPE model based on the Encoder-Decoder CNN architecture, InfPose, which can perform real-time on edge devices. We first built a lightweight Encoder-Decoder CNN backbone based on hardware-friendly inverted residual blocks. Secondly, we utilized three methods to improve the capability of InfPose, including decoupling associative embedding head, multi-scale supervision, and cross-modal knowledge distillation. In addition, we gathered a wild infrared human pose dataset to train and evaluate our methods. Experiment results show that the proposed model is more robust and has less latency when inference on edge GPU platforms in comparison to the prevailing mainstream models. The inference time for InfPose on Xavier NX was recorded as 27.7 ms (approximate to 37 fps) and maintained sufficient accuracy for use. This research can be applied to human-machine interaction of autonomous vehicles or intelligent robots in nighttime or other scenes with poor visibility.

关键词： Pose estimation Decoding Convolution Real-time systems Performance evaluation Image resolution Head deep learning for visual perception gesture posture and facial expressions multi-human pose estimation infrared vision

来源：评论

学校读者我要写书评

暂无评论

Multi-Camera Unified Pre-Training via 3D Scene Reconstruction

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第4期9卷 3243-3250页

作者： Min, Chen Xiao, Liang Zhao, Dawei Nie, Yiming Dai, Bin Peking Univ Sch Comp Sci Beijing 100871 Peoples R China Def Innovat Inst Unmanned Syst Technol Res Ctr Beijing 10073 Peoples R China Intelligent Game & Decis Lab Beijing 100071 Peoples R China

Multi-camera 3D perception has emerged as a prominent research field in autonomous driving, offering a viable and cost-effective alternative to LiDAR-based solutions. The existing multi-camera algorithms primarily rely on monocular 2D pre-training. However, the monocular 2D pre-training overlooks the spatial and temporal correlations among the multi-camera system. To address this limitation, we propose the first multi-camera unified pre-training framework, called UniScene, which involves initially reconstructing the 3D scene as the foundational stage and subsequently fine-tuning the model on downstream tasks. Specifically, we employ Occupancy as the general representation for the 3D scene, enabling the model to grasp geometric priors of the surrounding world through pre-training. A significant benefit of UniScene is its capability to utilize a considerable volume of unlabeled image-LiDAR pairs for pre-training purposes. The proposed multi-camera unified pre-training framework demonstrates promising results in key tasks such as multi-camera 3D object detection and surrounding semantic scene completion. When compared to monocular pre-training methods, UniScene shows a significant improvement of about 2.0% in mAP and 2.0% in NDS for multi-camera 3D object detection, as well as a 3% increase in mIoU for surrounding semantic scene completion. By adopting our unified pre-training method, a 25% reduction in 3D training annotation costs can be achieved, offering significant practical value for the implementation of real-world autonomous driving.

关键词： Autonomous vehicles Decoding Laser radar Autonomous vehicle navigation deep learning for visual perception recognition representation learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：