检索结果-内蒙古大学图书馆

Maximizing Self-Supervision From Thermal Image for Effective Self-Supervised learning of Depth and Ego-Motion

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第3期7卷 7771-7778页

作者： Shin, Ukcheol Lee, Kyunghyun Lee, Byeong-Uk Kweon, In So Korea Adv Inst Sci & Technol Sch Elect Engn Daejeon 34141 South Korea

Recently, self-supervised learning of depth and ego-motion from thermal images shows strong robustness and reliability under challenging scenarios. However, the inherent thermal image properties such as weak contrast, blurry edges, and noise hinder to generate effective self-supervision from thermal images. Therefore, most research relies on additional self-supervision sources such as well-lit RGB images, generative models, and Lidar information. In this letter, we conduct an in-depth analysis of thermal image characteristics that degenerates self-supervision from thermal images. Based on the analysis, we propose an effective thermal image mapping method that significantly increases image information, such as overall structure, contrast, and details, while preserving temporal consistency. The proposed method shows outperformed depth and pose results than previous state-of-the-art networks without leveraging additional RGB guidance.

关键词： Autonomous vehicle navigation computer vision for transportation deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

SGM3D: Stereo Guided Monocular 3D Object Detection

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第4期7卷 10478-10485页

作者： Zhou, Zheyuan Du, Liang Ye, Xiaoqing Zou, Zhikang Tan, Xiao Zhang, Li Xue, Xiangyang Feng, Jianfeng Fudan Univ Sch Comp Sci Shanghai 200437 Peoples R China Fudan Univ Inst Sci & Technol Brain Inspired Intelligence Shanghai 200437 Peoples R China Baidu Inc Dept Comp Vis Technol VIS Beijing 100085 Peoples R China Fudan Univ Sch Data Sci Shanghai 200437 Peoples R China

Monocular 3D object detection aims to predict the object location, dimension and orientation in 3D space alongside the object category given only a monocular image. It poses a great challenge due to its ill-posed property, which is a critical lack of depth information in the 2D image plane. While existing approaches leverage off-the-shelf depth estimation or rely on LiDAR sensors to mitigate this problem, dependence on the additional depth model or expensive equipment severely limits their scalability to generic 3D perception. In this paper, we propose a stereo-guided monocular 3D object detection framework, dubbed SGM3D, which aligns the monocular feature to the stereo feature and exploits the network's ability of generating the cross-modal feature for accurate 3D object detection. We present a multi-granularity feature alignment (MG-FA) mechanism to exploit the network's ability to generate stereo-mimicking features given only monocular cues. Coarse feature-level, as well as fine anchor-level feature alignment, are both leveraged for monocular feature guidance. In addition, we introduce an IoU matching-based feature alignment (IoU-MA) method for object-level feature alignment between the stereo and monocular predictions to alleviate the mismatches while adopting the MG-FA. Extensive experiments on the KITTI datasets demonstrate the effectiveness of our method. Code is publicly available at https://***/zhouzheyuan/sgm3d.

关键词： Autonomous vehicle navigation deep learning for visual perception monocular 3D object detection

来源：评论

学校读者我要写书评

暂无评论

ABCD: Attentive Bilateral Convolutional Network for Robust Depth Completion

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第1期7卷 81-87页

作者： Jeon, Yurim Kim, Hwichang Seo, Seung-Woo Seoul Natl Univ SNU Dept Elect & Comp Engn Seoul 08826 South Korea Inst New Media & Commun INMC Seoul 08826 South Korea

We propose a point-cloud-centric depth completion method called attention bilateral convolutional network for depth completion (ABCD). The proposed method uses LiDAR data and camera data to improve the resolution of the sparse depth information. Color images, which have been seen as fundamental to depth completion tasks, are inevitably sensitive to light and weather conditions. We designed an attentive bilateral convolutional layer (ABCL) to build a robust depth completion network under diverse environmental conditions. An ABCL efficiently learns geometric characteristics by directly leveraging a 3D point cloud and enhances the representation capability of sparse depth information by highlighting the core while suppressing clutter. The ABCD, with an ABCL as a building block, stably fills the void in sparse depth images even under unfamiliar conditions with minimum dependency on unstable camera sensors. Therefore, the proposed method is expected to be a solution to depth completion problems caused by changes in the environment in which images are captured. Through comparative experiments with other methods using the KITTI [1] and VirtualKITTI2 [2] datasets, we demonstrated the outstanding performance of the proposed method in diverse driving environments.

关键词： Three-dimensional displays Lattices Laser radar Feature extraction Convolution Task analysis Cameras Computer vision for automation deep learning for visual perception sensor fusion

来源：评论

学校读者我要写书评

暂无评论

Object-Aware Monocular Depth Prediction With Instance Convolutions

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第2期7卷 5389-5396页

作者： Simsar, Enis Ornek, Evin Pnar Manhardt, Fabian Dhamo, Helisa Navab, Nassir Tombari, Federico Tech Univ Munich D-80333 Munich Germany Google Inc D-85748 Garching Germany

With the advent of deep learning, estimating depth from a single RGB image has recently received a lot of attention, being capable of empowering many different applications ranging from path planning for robotics to computational cinematography. Nevertheless,while the depth maps are in their entirety fairly reliable, the estimates around object discontinuities are still far from satisfactory. This can be attributed to the fact that the convolutional operator naturally aggregates features across object discontinuities, resulting in smooth transitions rather than clear boundaries. Therefore, in order to circumvent this issue, we propose a novel convolutional operator which is explicitly tailored to avoid feature aggregation of different object parts. In particular, our method is based on estimating per-part depth values by means of super-pixels. The proposed convolutional operator, which we dub "Instance Convolution," then only considers each object part individually on the basis of the estimated super-pixels. Our evaluation with respect to the NYUv2, iBims and KITTI datasets demonstrate the advantages of Instance Convolutions over the classical convolution at estimating depth around occlusion boundaries, while producing comparable results elsewhere. Our code is available at ***/enisimsar/instance-conv.

关键词： deep learning for visual perception RGB-D perception

来源：评论

学校读者我要写书评

暂无评论

Few-Shot Instance Grasping of Novel Objects in Clutter

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第3期7卷 6566-6573页

作者： Guo, Weikun Li, Wei Hu, Ziye Gan, Zhongxue Fudan Univ Acad Engn & Technol Shanghai 200437 Peoples R China Ji Hua Lab Dept Engn Res Ctr Intelligent Robot Foshan 528200 Guangdong Peoples R China

Instance grasping, which aims to grasp a specific object out of clutter, is a fundamental task within robotics. However, allowing a robot to quickly learn to perform instance grasping for new, previously unseen objects remains challenging. In this work, we present an instance grasping meta-learning framework (IGML), a simple yet effective end-to-end approach that not only teaches robots to identify novel objects but also how to grasp them. Given only a few examples to specify the grasping point of the target object, our IGML can quickly learn to recognize the target object and grasp it at the demonstrated grasping point by leveraging prior experience. Experimental results on the test sets show that IGML achieves decent success rates in cluttered environments, significantly surpassing state-of-the-art methods. Then we deployed IGML on a UR5 robot arm to handle pick-and-place scenarios and achieved a precision rate of 93.4% and a recall rate of 87.1%.

关键词： deep learning for visual perception deep learning in grasping and manipulation perception for grasping and manipulation

来源：评论

学校读者我要写书评

暂无评论

See Eye to Eye: A Lidar-Agnostic 3D Detection Framework for Unsupervised Multi-Target Domain Adaptation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第3期7卷 7904-7911页

作者： Tsai, Darren Berrio, Julie Stephany Shan, Mao Worrall, Stewart Nebot, Eduardo Univ Sydney Australian Ctr Field Robot ACFR Sydney NSW Australia

Sampling discrepancies between different manufacturers and models of lidar sensors result in inconsistent representations of objects. This leads to performance degradation when 3D detectors trained for one lidar are tested on other types of lidars. Remarkable progress in lidar manufacturing has brought about advances in mechanical, solid-state, and recently, adjustable scan pattern lidars. For the latter, existing works often require fine-tuning the model each time scan patterns are adjusted, which is infeasible. We explicitly deal with the sampling discrepancy by proposing a novel unsupervised multi-target domain adaptation framework, SEE, for transferring the performance of state-of-the-art 3D detectors across both fixed and flexible scan pattern lidars without requiring fine-tuning of models by end-users. Our approach interpolates the underlying geometry and normalises the scan pattern of objects from different lidars before passing them to the detection network. We demonstrate the effectiveness of SEE on public datasets, achieving state-of-the-art results, and additionally provide quantitative results on a novel high-resolution lidar to prove the industry applications of our framework.

关键词： Object detection segmentation and categorization deep learning for visual perception recognition

来源：评论

学校读者我要写书评

暂无评论

Multi-Level Consistency learning for Source-Free Model Adaptation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第4期7卷 12419-12426页

作者： Luo, Xin Chen, Wei Li, Chen Zhou, Bin Tan, Yusong Natl Univ Def Technol Coll Comp Sci Changsha 410000 Peoples R China

Source-free model adaptation (SFMA) plays an important role in robust robot automation, which aims to mitigate distributional inconsistency between source and target data while avoiding accessing source data. SFMA methods generally benefit from self-training, which can be prone to overfitting noisy pseudo labels. Previous methods used various techniques to filter out noisy predictions and improve self-training performance. Nonetheless, the filtering is heavily reliant on pre-defined thresholds and suffers from a class imbalance problem, resulting in a bias towards the majority classes and poor performance of the other classes. Aiming at this pitfall, this study proposes to combat the noisy pseudo labels via multi-level perturbation, which involves all the pseudo predictions in self-training. The proposed method introduces different perturbations at three different levels: input, feature, and model. These perturbations boost the diversity of training data and increase the difficulty of fitting pseudo labels, which avoids overfitting the noise without ignoring the minority classes. Besides, the proposed method helps mitigate the intra-domain gap by maximizing the consistency between the original predictions and the perturbed predictions. Experimental results on two domain adaptive segmentation benchmarks confirm the effectiveness of the proposed method, which outperforms state-of-the-art SFMA methods.

关键词： Computer vision for automation deep learning for visual perception transfer learning

来源：评论

学校读者我要写书评

暂无评论

Temporal Point Cloud Completion With Pose Disturbance

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第2期7卷 4165-4172页

作者： Shi, Jieqi Xu, Lingyun Li, Peiliang Chen, Xiaozhi Shen, Shaojie Hong Kong Univ Sci & Technol Dept Elect & Comp Engn Hong Kong Peoples R China Dji Co Shenzhen 518057 Peoples R China

Point clouds collected by real-world sensors are always unaligned and sparse, which makes it hard to reconstruct the complete shape of object from a single frame of data. In this work, we manage to provide complete point clouds from sparse input with pose disturbance by limited translation and rotation. We also use temporal information to enhance the completion model, refining the output with a sequence of inputs. With the help of gated recovery units(GRU) and attention mechanisms as temporal units, we propose a point cloud completion framework that accepts a sequence of unaligned and sparse inputs, and outputs consistent and aligned point clouds. Our network performs in an online manner and presents a refined point cloud for each frame, which enables it to be integrated into any SLAM or reconstruction pipeline. As far as we know, our framework is the first to utilize temporal information and ensure temporal consistency with limited transformation. Through experiments in ShapeNet and KITTI, we prove that our framework is effective in both synthetic and real-world datasets.

关键词： Point cloud compression Three-dimensional displays Shape Task analysis Pipelines Solid modeling Sensors Artificial intelligence deep learning for visual perception 3D completion

来源：评论

学校读者我要写书评

暂无评论

EchoVPR: Echo State Networks for visual Place Recognition

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第2期7卷 4520-4527页

作者： Ozdemir, Anil Scerri, Mark Barron, Andrew B. Philippides, Andrew Mangan, Michael Vasilaki, Eleni Manneschi, Luca Univ Sheffield Dept Comp Sci Sheffield S1 4DP S Yorkshire England Macquarie Univ Dept Biol Sci Sydney NSW 2109 Australia Univ Sussex Sch Engn & Informat Brighton BN19QJ E Sussex England Univ Sheffield Dept Comp Sci Sheffield S1 4DP S Yorkshire England Univ Zurich Inst Neuroinformat CH-8057 Zurich Switzerland Swiss Fed Inst Technol CH-8057 Zurich Switzerland

Recognising previously visited locations is an important, but unsolved, task in autonomous navigation. Current visual place recognition (VPR) benchmarks typically challenge models to recover the position of a query image (or images) from sequential datasets that include both spatial and temporal components. Recently, Echo State Network (ESN) varieties have proven particularly powerful at solving machine learning tasks that require spatio-temporal modelling. These networks are simple, yet powerful neural architectures that-exhibiting memory over multiple time-scales and non-linear high-dimensional representations-can discover temporal relations in the data while still maintaining linearity in the learning time. In this letter, we present a series of ESNs and analyse their applicability to the VPR problem. We report that the addition of ESNs to pre-processed convolutional neural networks led to a dramatic boost in performance in comparison to non-recurrent networks in five out of six standard benchmarks (GardensPoint, SPEDTest, ESSEX3IN1, Oxford RobotCar, and Nordland), demonstrating that ESNs are able to capture the temporal structure inherent in VPR problems. Moreover, we show that models that include ESNs can outperform class-leading VPR models which also exploit the sequential dynamics of the data. Finally, our results demonstrate that ESNs improve generalisation abilities, robustness, and accuracy further supporting their suitability to VPR applications.

关键词： Vision-based navigation deep learning for visual perception visual learning

来源：评论

学校读者我要写书评

暂无评论

Uncertainty-Aware Self-Improving Framework for Depth Estimation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第1期7卷 41-48页

作者： Nie, Xinyu Shi, Dianxi Li, Ruihao Liu, Zhe Chen, Xucan Artificial Intelligence Res Ctr Def Innovat Inst Beijing 100166 Peoples R China Tianjin Artificial Intelligence Innovat Ctr Tianjin 300457 Peoples R China Natl Univ Def Technol Coll Comp Changsha 410073 Peoples R China

In recent years, self-supervised paradigms for depth estimation have drawn lots of attention from the community. Promising as they are, in order to achieve wider application and better performance, reasoning about the uncertainty of the estimated depth maps is in great demand. In this paper, we propose a novel uncertainty-aware self-improving framework for self-supervised depth estimation, which could model and exploit uncertainty to boost depth accuracy without any external supervision. In detail, our framework includes two DepthNets-a teacher network and a student network. The teacher network is pre-trained in a self-supervised manner and the predicted depth is leveraged as pseudo-ground-truth to supervise the learning of the student network, leading to better estimation accuracy and the ability of modeling depth uncertainty. Afterward, the estimated uncertainty would be introduced back into the teacher network as the mask, preventing erroneous regions from contributing to network training and depth estimation. In this way, a better pseudo-ground-truth from the teacher network will be used for the next iteration of the student network training. We evaluate our proposed framework on the widely used KITTI dataset and TUM RGB-D dataset with both quantitative and qualitative experiments. The experimental results show that our framework achieves quite satisfactory performance among the self-supervised depth estimation methods. Extensive experiments which combine the student network and feature-based monocular SLAM system (i.e ORB-SLAM3) further demonstrate the benefits of the introduced depth uncertainty.

关键词： Uncertainty Estimation Simultaneous localization and mapping Training Task analysis deep learning Cameras deep learning methods deep learning for visual perception SLAM

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：