检索结果-内蒙古大学图书馆

learning Rearrangement Manipulation via Scene Prediction in Point Cloud

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第12期9卷 11090-11097页

作者： Ma, Anji Duan, Xingguang Beijing Inst Technol Sch Mechatron Engn Beijing 100081 Peoples R China

Predicting scene evolution conditioned on robotic actions is a vital technique in modeling robot manipulations. Previous studies have primarily focused on learning spatiotemporally continuous actions like Cartesian displacements, thus applying them to planar-pushing tasks. In this letter, we propose a scene prediction model that learns higher-level robot actions, such as grasping and pick-and-place, and is applicable to planning such actions in rearrangement manipulations. The model takes partially observed point clouds (e.g., from a single camera) and robot pick-and-place actions to predict point clouds of future scenes. The model directly learns scene prediction using point cloud observation and representation, without requiring prior knowledge of object properties like cad models, pose, or instance segmentation. We train the model only on a synthetic dataset acquired entirely automatically with minimal human intervention. Our experiments validate that our model can substantially learn robot grasping and pick-and-place actions and show that, when integrated into a sample-based planning framework, predicting scenes in point clouds outperforms the image-based baseline in grasping and rearrangement manipulation. Moreover, results show that our method can be directly transferred to real-world environments without fine-tuning and show promising performance on a collection of 18 household objects.

关键词： deep learning in grasping and manipulation manipulation planning deep learning for visual perception deep learning in grasping and manipulation manipulation planning

来源：评论

学校读者我要写书评

暂无评论

Collaborative Multi-Object Tracking With Conformal Uncertainty Propagation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第4期9卷 3323-3330页

作者： Su, Sanbao Han, Songyang Li, Yiming Zhang, Zhili Feng, Chen Ding, Caiwen Miao, Fei Univ Connecticut Dept Comp Sci & Engn Storrs CT 06268 USA Univ Connecticut Storrs CT 06268 USA NYU Tandon Sch Engn Brooklyn NY 11201 USA

Object detection and multiple object tracking (MOT) are essential components of self-driving systems. Accurate detection and uncertainty quantification are both critical for onboard modules, such as perception, prediction, and planning, to improve the safety and robustness of autonomous vehicles. Collaborative object detection (COD) has been proposed to improve detection accuracy and reduce uncertainty by leveraging the viewpoints of multiple agents. However, little attention has been paid to how to leverage the uncertainty quantification from COD to enhance MOT performance. In this letter, as the first attempt to address this challenge, we design an uncertainty propagation framework called MOT-CUP. Our framework first quantifies the uncertainty of COD through direct modeling and conformal prediction, and propagates this uncertainty information into the motion prediction and association steps. MOT-CUP is designed to work with different collaborative object detectors and baseline MOT algorithms. We evaluate MOT-CUP on V2X-Sim, a comprehensive collaborative perception dataset, and demonstrate a 2% improvement in accuracy and a 2.67 x reduction in uncertainty compared to the baselines, e.g. SORT and ByteTrack. In scenarios characterized by high occlusion levels, our MOT-CUP demonstrates a noteworthy 4.01% improvement in accuracy. MOT-CUP demonstrates the importance of uncertainty quantification in both COD and MOT, and provides the first attempt to improve the accuracy and reduce the uncertainty in MOT based on COD through uncertainty propagation.

关键词： Uncertainty Object detection Collaboration Kalman filters Prediction algorithms Predictive models Estimation Computer vision for transportation deep learning for visual perception multi-robot systems visual tracking

来源：评论

学校读者我要写书评

暂无评论

learning-To-Rank Approach for Identifying Everyday Objects Using a Physical-World Search Engine

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第3期9卷 2088-2095页

作者： Kaneda, Kanta Nagashima, Shunya Korekata, Ryosuke Kambara, Motonari Sugiura, Komei Keio Univ Yokohama 2238522 Japan

Domestic service robots offer a solution to the increasing demand for daily care and support. A human-in-the-loop approach that combines automation and operator intervention is considered to be a realistic approach to their use in society. Therefore, we focus on the task of retrieving target objects from open-vocabulary user instructions in a human-in-the-loop setting, which we define as the learning-to-rank physical objects (LTRPO) task. For example, given the instruction "Please go to the dining room which has a round table. Pick up the bottle on it," the model is required to output a ranked list of target objects that the operator/user can select. In this letter, we propose MultiRankIt, which is a novel approach for the LTRPO task. MultiRankIt introduces the Crossmodal Noun Phrase Encoder to model the relationship between phrases that contain referring expressions and the target bounding box, and the Crossmodal Region Feature Encoder to model the relationship between the target object and multiple images of its surrounding contextual environment. Additionally, we built a new dataset for the LTRPO task that consists of instructions with complex referring expressions accompanied by real indoor environmental images that feature various target objects. We validated our model on the dataset and it outperformed the baseline method in terms of the mean reciprocal rank and recall@k. Furthermore, we conducted physical experiments in a setting where a domestic service robot retrieved everyday objects in a standardized domestic environment, based on users' instruction in a human-in-the-loop setting. The experimental results demonstrate that the success rate for object retrieval achieved 80%.

关键词： deep learning methods learning categories and concepts deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

Delta Descriptors: Change-Based Place Representation for Robust visual Localization

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2020年第4期5卷 5120-5127页

作者： Garg, Sourav Harwood, Ben Anand, Gaurangi Milford, Michael Queensland Univ Technol Sch Elect Engn & Robot Brisbane Qld Australia Australian Ctr Robot Vis Brisbane Qld 4001 Australia QUT Ctr Robot Brisbane Qld 4001 Australia Monash Univ Dept Elect & Comp Syst Engn Clayton Vic 3800 Australia Queensland Univ Technol Sch Comp Sci Brisbane Qld 4001 Australia

visual place recognition is challenging because there are so many factors that can cause the appearance of a place to change, from day-night cycles to seasonal change to atmospheric conditions. In recent years a large range of approaches have been developed to address this challenge including deep-learnt image descriptors, domain translation, and sequential filtering, all with shortcomings including generality and velocity-sensitivity. In this letter we propose a novel descriptor derived from tracking changes in any learned global descriptor over time, dubbed Delta Descriptors. Delta Descriptors mitigate the offsets induced in the original descriptor matching space in an unsupervised manner by considering temporal differences across places observed along a route. Like all other approaches, Delta Descriptors have a shortcoming - volatility on a frame to frame basis - which can be overcome by combining them with sequential filtering methods. Using two benchmark datasets, we first demonstrate the high performance of Delta Descriptors in isolation, before showing new state-of-the-art performance when combined with sequence-based matching. We also present results demonstrating the approach working with four different underlying descriptor types, and two other beneficial properties of Delta Descriptors in comparison to existing techniques: their increased inherent robustness to variations in camera motion and a reduced rate of performance degradation as dimensional reduction is applied. Source code is made available at https://***/oravus/DeltaDescriptors.

关键词： Localization deep learning for visual perception recognition computer vision for automation autonomous vehicle navigation

来源：评论

学校读者我要写书评

暂无评论

DiffMap: Enhancing Map Segmentation With Map Prior Using Diffusion Model

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第11期9卷 9836-9843页

作者： Jia, Peijin Wen, Tuopu Luo, Ziang Yang, Mengmeng Jiang, Kun Liu, Ziyuan Tang, Xuewei Lei, Zhiquan Cui, Le Zhang, Bo Sheng, Kehua Yang, Diange Tsinghua Univ Sch Vehicle & Mobil Beijing 100084 Peoples R China DiDi Chuxing Tianjin 300450 Peoples R China

Constructing high-definition (HD) maps is a crucial requirement for enabling autonomous driving. In recent years, several map segmentation algorithms have been developed to address this need, leveraging advancements in Bird's-Eye View (BEV) perception. However, existing models still encounter challenges in producing realistic and consistent semantic map layouts. A prominent issue is the limited utilization of structured priors inherent in map segmentation masks. In light of this, we propose DiffMap, a novel approach specifically designed to model the structured priors of map segmentation masks using latent diffusion model. By incorporating this technique, the performance of existing semantic segmentation methods can be significantly enhanced and certain structural errors present in the segmentation outputs can be effectively rectified. Notably, the proposed module can be seamlessly integrated into any map segmentation model, thereby augmenting its capability to accurately delineate semantic information. Furthermore, through extensive visualization analysis, our model demonstrates superior proficiency in generating results that more accurately reflect real-world map layouts, further validating its efficacy in improving the quality of the generated maps.

关键词： Semantics Diffusion models Noise reduction Predictive models Laser radar Standards Noise Autonomous vehicle navigation deep learning for visual perception sensor fusion

来源：评论

学校读者我要写书评

暂无评论

DenseLiDAR: A Real-Time Pseudo Dense Depth Guided Depth Completion Network

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第2期6卷 1808-1815页

作者： Gu, Jiaqi Xiang, Zhiyu Ye, Yuwen Wang, Lingxuan Zhejiang Univ Coll Informat Sci & Eletron Engn Hangzhou 310027 Peoples R China Zhejiang Univ Zhejiang Prov Key Lab Informat Proc Commun & Netw Hangzhou 310027 Peoples R China

Depth Completion can produce a dense depth map from a sparse input and provide a more complete 3D description of the environment. Despite great progress made in depth completion, the sparsity of the input and low density of the ground truth still make this problem challenging. In this work, we propose DenseLiDAR, a novel real-time pseudo-depth guided depth completion neural network. We exploit dense pseudo-depth map obtained from simple morphological operations to guide the network in three aspects: (1) Constructing a residual structure for the output;(2) Rectifying the sparse input data;(3) Providing dense structural loss for training the network. Thanks to these novel designs, higher performance of the output could be achieved. In addition, two new metrics for better evaluating the quality of the predicted depth map are also presented. Extensive experiments on KITTI depth completion benchmark suggest that our model is able to achieve the state-of-the-art performance at the highest frame rate of 50 Hz. The predicted dense depth is further evaluated by several downstream robotic perception or positioning tasks. For the task of 3D object detection, 3(similar to)5 percent performance gains on small objects categories are achieved on KITTI 3D object detection dataset. For RGB-D SLAM, higher accuracy on vehicle's trajectory is also obtained in KITTI Odometry dataset. These promising results not only verify the high quality of our depth prediction, but also demonstrate the potential of improving the related downstream tasks by using depth completion results.

关键词： deep learning for visual perception RGB-D perception recognition

来源：评论

学校读者我要写书评

暂无评论

GA-Nav: Efficient Terrain Segmentation for Robot Navigation in Unstructured Outdoor Environments

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第3期7卷 8138-8145页

作者： Guan, Tianrui Kothandaraman, Divya Chandra, Rohan Sathyamoorthy, Adarsh Jagan Weerakoon, Kasun Manocha, Dinesh Univ Maryland Dept Comp Sci College Pk MD 20742 USA Univ Maryland Dept Elect & Comp Engn College Pk MD 20742 USA

We propose GA-Nav, a novel group-wise attention mechanism to identify safe and navigable regions in unstructured environments from RGB images. Our group-wise attention method extracts multi-scale features from each type of terrain independently and classifies terrains based on their navigability levels using coarse-grained semantic segmentation. Our novel loss can be embedded within any backbone network to explicitly focus on the different groups' features, at a low spatial resolution. Our design leads to efficient inference while maintaining a high level of accuracy compared to existing SOTA methods. Our extensive evaluations on the RUGD and RELLIS-3D datasets shows that GA-Nav achieves the state-of-the-art performance on RUGD and RELLIS-3D datasets. We interface GA-Nav with a deep reinforcement learning-based navigation algorithm and highlight its benefits in terms of navigation in real-world unstructured terrains. We integrate our GA-Nav-based navigation algorithm with ClearPath Jackal and Husky robots, and observe an improvement in terms of navigation success rate and better trajectory selections.

关键词： Vision-based navigation AI-enabled robotics deep learning for visual perception AI-based methods deep learning methods

来源：评论

学校读者我要写书评

暂无评论

SE3ET: SE(3)-Equivariant Transformer for Low-Overlap Point Cloud Registration

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第11期9卷 9526-9533页

作者： Lin, Chien Erh Zhu, Minghan Ghaffari, Maani Univ Michigan Ann Arbor MI 48109 USA

Partial point cloud registration is a challenging problem in robotics, especially when the robot undergoes a large transformation, causing a significant initial pose error and a low overlap between measurements. This letter proposes exploiting equivariant learning from 3D point clouds to improve registration robustness. We propose SE3ET, an SE(3)-equivariant registration framework that employs equivariant point convolution and equivariant transformer designs to learn expressive and robust geometric features. We tested the proposed registration method on indoor and outdoor benchmarks where the point clouds are under arbitrary transformations and low overlapping ratios. We also provide generalization tests and run-time performance.

关键词： Point cloud compression Transformers Feature extraction Three-dimensional displays Representation learning Convolution Decoding deep learning for visual perception localization deep learning methods

来源：评论

学校读者我要写书评

暂无评论

OPA-3D: Occlusion-Aware Pixel-Wise Aggregation for Monocular 3D Object Detection

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2023年第3期8卷 1327-1334页

作者： Su, Yongzhi Di, Yan Zhai, Guangyao Manhardt, Fabian Rambach, Jason Busam, Benjamin Stricker, Didier Tombari, Federico German Res Ctr Artificial Intelligence DFKI GmbH D-67663 Kaiserslautern Germany TU Kaiserslautern Fac Comp Sci D-67663 Kaiserslautern Germany Tech Univ Munich Fac Comp Sci D-85748 Garching Germany Google CH-8002 Zurich Switzerland

Monocular 3D object detection has recently made a significant leap forward thanks to the use of pre-trained depth estimators for pseudo-LiDAR recovery. Yet, such two-stage methods typically suffer from overfitting and are incapable of explicitly encapsulating the geometric relation between depth and object bounding box. To overcome this limitation, we instead propose to jointly estimate dense scene depth with depth-bounding box residuals and object bounding boxes, allowing a two-stream detection of 3D objects that harnesses both geometry and context information. Thereby, the geometry stream combines visible depth and depth-bounding box residuals to recover the object bounding box via explicit occlusion-aware optimization. In addition, a bounding box based geometry projection scheme is employed in an effort to enhance distance perception. The second stream, named as the Context Stream, directly regresses 3D object location and size. This novel two-stream representation enables us to enforce cross-stream consistency terms, which aligns the outputs of both streams, and further improves the overall performance. Extensive experiments on the public benchmark demonstrate that OPA-3D outperforms state-of-the-art methods on the main Car category, whilst keeping a real-time inference speed.

关键词： Three-dimensional displays Geometry Streaming media Feature extraction Shape Head Estimation Computer vision for transportation deep learning for visual perception object detection

来源：评论

学校读者我要写书评

暂无评论

Graph-Guided Deformation for Point Cloud Completion

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第4期6卷 7081-7088页

作者： Shi, Jieqi Xu, Lingyun Heng, Liang Shen, Shaojie Hong Kong Univ Sci & Technol Dept Elect & Comp Engn Hong Kong 999961 Peoples R China Dji Co Shenyang 110000 Liaoning Peoples R China

For a long time, the point cloud completion task has been regarded as a pure generation task. After obtaining the global shape code through the encoder, a complete point cloud is generated using the shape priorly learnt by the networks. However, such models are undesirably biased towards prior average objects and inherently limited to fit geometry details. In this letter, we propose a Graph-Guided Deformation Network, which respectively regards the input data and intermediate generation as controlling and supporting points, and models the optimization guided by a graph convolutional network(GCN) for the point cloud completion task. Our key insight is to simulate the least square Laplacian deformation process via mesh deformation methods, which brings adaptivity for modeling variation in geometry details. By this means, we also reduce the gap between the completion task and the mesh deformation algorithms. As far as we know, we are the first to refine the point cloud completion task by mimicing traditional graphics algorithms with GCN-guided deformation. We have conducted extensive experiments on both the simulated indoor dataset ShapeNet, outdoor dataset KITTI, and our self-collected autonomous driving dataset Pandar40. The results show that our method outperforms the existing state-of-the-art algorithms in the 3D point cloud completion task.

关键词： deep learning for visual perception graph neural network 3D completion

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：