检索结果-内蒙古大学图书馆

SBAS: Salient Bundle Adjustment for visual SLAM

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT 2021年 70卷 1页

作者： Wang, Ke Ma, Sai Ren, Fan Lu, Jianbo Chongqing Univ Coll Mech & Vehicle Engn Chongqing 400044 Peoples R China Chongqing Univ Key Lab Mech Transmiss Chongqing 400044 Peoples R China Changan Automobile Co Intelligent Vehicle Res & Dev Inst Chongqing 401120 Peoples R China Ford Motor Co Res & Adv Engn Dearborn MI 48121 USA

Most visual simultaneous localization and mapping (SLAM) algorithms rely on geometric features, such as points, lines, and planes, and these algorithms treated all features as equally important in the optimization process, ignoring the semantic information of these features. However, the robust and rich semantic features should play a more important role, which is similar to the mechanism of visual saliency or visual attention. Therefore, this article aims to mimic this mechanism in the visual SLAM framework by using the saliency prediction model. To solve the center bias of the generic salient dataset, we proposed a method for recomputing saliency map by considering both geometric and semantic information. Then, we proposed the salient bundle adjustment (SBA) algorithm by using the value of saliency prediction map as the weight of the feature points in the traditional bundle adjustment (BA) approach. Finally, exhaustive experiments conducted with the state-of-the-art algorithm show that our proposed algorithm outperforms existing algorithms such as direct sparse odometry (DSO) and oriented brief (ORB)SLAM3 in both indoor and outdoor environments.

关键词： deep learning for visual perception saliency prediction simultaneous localization and mapping (SLAM)

来源：评论

学校读者我要写书评

暂无评论

On the Challenges of Open World Recognition Under Shifting visual Domains

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第2期6卷 604-611页

作者： Fontanel, Dario Cermelli, Fabio Mancini, Massimiliano Caputo, Barbara Politecn Torino DAUIN I-10138 Turin Italy Italian Inst Technol I-16163 Genoa Italy Univ Tubingen D-72074 Tubingen Germany

Robotic visual systems operating in the wild must act in unconstrained scenarios, under different environmental conditions while facing a variety of semantic concepts, including unknown ones. To this end, recent works tried to empower visual object recognition methods with the capability to i) detect unseen concepts and ii) extended their knowledge over time, as images of new semantic classes arrive. This setting, called Open World Recognition (OWR), has the goal to produce systems capable of breaking the semantic limits present in the initial training set. However, this training set imposes to the system not only its own semantic limits, but also environmental ones, due to its bias toward certain acquisition conditions that do not necessarily reflect the high variability of the real-world. This discrepancy between training and test distribution is called domain-shift. This letter investigates whether OWR algorithms are effective under domain-shift, presenting the first benchmark setup for assessing fairly the performances of OWR algorithms, with and without domain-shift. We then use this benchmark to conduct analyses in various scenarios, showing how existing OWR algorithms indeed suffer a severe performance degradation when train and test distributions differ. Our analysis shows that this degradation is only slightly mitigated by coupling OWR with domain generalization techniques, indicating that the mere plug-and-play of existing algorithms is not enough to recognize new and unknown categories in unseen domains. Our results clearly point toward open issues and future research directions, that need to be investigated for building robot visual systems able to function reliably under these challenging yet very real conditions.

关键词： Training Robots Semantics Feature extraction Benchmark testing visualization visual systems deep learning for visual perception visual learning recognition

来源：评论

学校读者我要写书评

暂无评论

Multi-GAT: A Graphical Attention-Based Hierarchical Multimodal Representation learning Approach for Human Activity Recognition

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第2期6卷 1729-1736页

作者： Islam, Md Mofijul Iqbal, Tariq Univ Virginia Sch Engn & Appl Sci Charlottesville VA 22903 USA

Recognizing human activities is one of the crucial capabilities that a robot needs to have to be useful around people. Although modern robots are equipped with various types of sensors, human activity recognition (HAR) still remains a challenging problem, particularly in the presence of noisy sensor data. In this work, we introduce a multimodal graphical attention-based HAR approach, called Multi-GAT, which hierarchically learns complementary multimodal features. We develop a multimodal mixture-of-experts model to disentangle and extract salient modality-specific features that enable feature interactions. Additionally, we introduce a novel message-passing based graphical attention approach to capture cross-modal relation for extracting complementary multimodal features. The experimental results on two multimodal human activity datasets suggest that Multi-GAT outperformed state-of-the-art HAR algorithms across all datasets and metrics tested. Finally, the experimental results with noisy sensor data indicate that Multi-GAT consistently outperforms all the evaluated baselines. The robust performance suggests that Multi-GAT can enable seamless human-robot collaboration in noisy human environments.

关键词： deep learning for visual perception gesture multi modal perception for HRI posture and facial expressions

来源：评论

学校读者我要写书评

暂无评论

MOLTR: Multiple Object Localization, Tracking and Reconstruction From Monocular RGB Videos

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第2期6卷 3341-3348页

作者： Li, Kejie Rezatofighi, Hamid Reid, Ian Univ Adelaide Sch Comp Sci Adelaide SA 5005 Australia Australian Ctr Robot Vis Adelaide SA 5005 Australia Monash Univ Fac Informat Technol Dept Data Sci & AI Melbourne Vic 3800 Australia

Semantic aware reconstruction is more advantageous than geometric-only reconstruction for future robotic and AR/VR applications because it represents not only where things are, but also what things are. Object-centric mapping is a task to build an object-level reconstruction where objects are separate and meaningful entities that convey both geometry and semantic information. In this letter, we present MOLTR, a solution to object-centric mapping using only monocular image sequences and camera poses. It is able to localize, track and reconstruct multiple rigid objects in an online fashion when a RGB camera captures a video of the surrounding. Given a new RGB frame, MOLTR firstly applies a monocular 3D detector to localize objects of interest and extract their shape codes that represent the object shape in a learnt embedding space. Detections are then merged to existing objects in the map after data association. Motion state (i.e., kinematics and the motion status) of each object is tracked by a multiple model Bayesian filter and object shape is progressively refined by fusing multiple shape code. We evaluate localization, tracking and reconstruction on benchmarking datasets for indoor and outdoor scenes, and show superior performance over previous approaches.

关键词： Mapping deep learning for visual perception recognition

来源：评论

学校读者我要写书评

暂无评论

Graph-Guided Deformation for Point Cloud Completion

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第4期6卷 7081-7088页

作者： Shi, Jieqi Xu, Lingyun Heng, Liang Shen, Shaojie Hong Kong Univ Sci & Technol Dept Elect & Comp Engn Hong Kong 999961 Peoples R China Dji Co Shenyang 110000 Liaoning Peoples R China

For a long time, the point cloud completion task has been regarded as a pure generation task. After obtaining the global shape code through the encoder, a complete point cloud is generated using the shape priorly learnt by the networks. However, such models are undesirably biased towards prior average objects and inherently limited to fit geometry details. In this letter, we propose a Graph-Guided Deformation Network, which respectively regards the input data and intermediate generation as controlling and supporting points, and models the optimization guided by a graph convolutional network(GCN) for the point cloud completion task. Our key insight is to simulate the least square Laplacian deformation process via mesh deformation methods, which brings adaptivity for modeling variation in geometry details. By this means, we also reduce the gap between the completion task and the mesh deformation algorithms. As far as we know, we are the first to refine the point cloud completion task by mimicing traditional graphics algorithms with GCN-guided deformation. We have conducted extensive experiments on both the simulated indoor dataset ShapeNet, outdoor dataset KITTI, and our self-collected autonomous driving dataset Pandar40. The results show that our method outperforms the existing state-of-the-art algorithms in the 3D point cloud completion task.

关键词： deep learning for visual perception graph neural network 3D completion

来源：评论

学校读者我要写书评

暂无评论

DenseLiDAR: A Real-Time Pseudo Dense Depth Guided Depth Completion Network

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第2期6卷 1808-1815页

作者： Gu, Jiaqi Xiang, Zhiyu Ye, Yuwen Wang, Lingxuan Zhejiang Univ Coll Informat Sci & Eletron Engn Hangzhou 310027 Peoples R China Zhejiang Univ Zhejiang Prov Key Lab Informat Proc Commun & Netw Hangzhou 310027 Peoples R China

Depth Completion can produce a dense depth map from a sparse input and provide a more complete 3D description of the environment. Despite great progress made in depth completion, the sparsity of the input and low density of the ground truth still make this problem challenging. In this work, we propose DenseLiDAR, a novel real-time pseudo-depth guided depth completion neural network. We exploit dense pseudo-depth map obtained from simple morphological operations to guide the network in three aspects: (1) Constructing a residual structure for the output;(2) Rectifying the sparse input data;(3) Providing dense structural loss for training the network. Thanks to these novel designs, higher performance of the output could be achieved. In addition, two new metrics for better evaluating the quality of the predicted depth map are also presented. Extensive experiments on KITTI depth completion benchmark suggest that our model is able to achieve the state-of-the-art performance at the highest frame rate of 50 Hz. The predicted dense depth is further evaluated by several downstream robotic perception or positioning tasks. For the task of 3D object detection, 3(similar to)5 percent performance gains on small objects categories are achieved on KITTI 3D object detection dataset. For RGB-D SLAM, higher accuracy on vehicle's trajectory is also obtained in KITTI Odometry dataset. These promising results not only verify the high quality of our depth prediction, but also demonstrate the potential of improving the related downstream tasks by using depth completion results.

关键词： deep learning for visual perception RGB-D perception recognition

来源：评论

学校读者我要写书评

暂无评论

visual-Tactile Cross-Modal Data Generation Using Residue-Fusion GAN With Feature-Matching and Perceptual Losses

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第4期6卷 7525-7532页

作者： Cai, Shaoyu Zhu, Kening Ban, Yuki Narumi, Takuji City Univ Hong Kong Sch Creat Media Hong Kong Peoples R China City Univ Hong Kong Shenzhen Res Inst Shenzhen Peoples R China Univ Tokyo Grad Sch Frontier Sci Chiba Japan Univ Tokyo Grad Sch Informat Sci & Technol Tokyo Japan JST PRESTO Tokyo Japan

Existing psychophysical studies have revealed that the cross-modal visual-tactile perception is common for humans performing daily activities. However, it is still challenging to build the algorithmic mapping from one modality space to another, namely the cross-modal visual-tactile data translation/generation, which could be potentially important for robotic operation. In this letter, we propose a deep-learning-based approach for cross-modal visual-tactile data generation by leveraging the framework of the generative adversarial networks (GANs). Our approach takes the visual image of a material surface as the visual data, and the accelerometer signal induced by the pen-sliding movement on the surface as the tactile data. We adopt the conditional-GAN (cGAN) structure together with the residue-fusion (RF) module, and train the model with the additional feature-matching (FM) and perceptual losses to achieve the cross-modal data generation. The experimental results show that the inclusion of the RF module, and the FM and the perceptual losses significantly improves cross-modal data generation performance in terms of the classification accuracy upon the generated data and the visual similarity between the ground-truth and the generated data.

关键词： visual learning deep learning for visual perception haptics and haptic interfaces

来源：评论

学校读者我要写书评

暂无评论

State-Consistency Loss for learning Spatial perception Tasks From Partial Labels

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第2期6卷 1112-1119页

作者： Nava, Mirko Gambardella, Luca Maria Giusti, Alessandro Dalle Molle Inst Artificial Intelligence IDSIA USI SUPSI CH-6928 Lugano Switzerland

When learning models for real-world robot spatial perception tasks, one might have access only to partial labels: this occurs for example in semi-supervised scenarios (in which labels are not available for a subset of the training instances) or in some types of self-supervised robot learning (where the robot autonomously acquires a labeled training set, but only acquires labels for a subset of the output variables in each instance). We introduce a general approach to deal with this class of problems using an auxiliary loss enforcing the expectation that the perceived environment state should not abruptly change;then, we instantiate the approach to solve two robot perception problems: a simulated ground robot learning long-range obstacle mapping as a 400-binary-label classification task in a self-supervised way in a static environment;and a real nano-quadrotor learning human pose estimation as a 3-variable regression task in a semi-supervised way in a dynamic environment. In both cases, our approach yields significant quantitative performance improvements (average increase of 6 AUC percentage points in the former;relative improvement of the R-2 metric ranging from 7% to 33% in the latter) over baselines.

关键词： Robots Task analysis Robot sensing systems Training Sensors Predictive models Robot vision systems deep learning methods deep learning for visual perception developmental robotics

来源：评论

学校读者我要写书评

暂无评论

Endo-Depth-and-Motion: Reconstruction and Tracking in Endoscopic Videos Using Depth Networks and Photometric Constraints

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第4期6卷 7225-7232页

作者： Recasens, David Lamarca, Jose Facil, Jose M. Montiel, J. M. M. Civera, Javier Univ Zaragoza I3A Zaragoza 50009 Spain

Estimating a scene reconstruction and the camera motion from in-body videos is challenging due to several factors, e.g. the deformation of in-body cavities or the lack of texture. In this paperwe present Endo-Depth-and-Motion, a pipeline that estimates the 6-degrees-of-freedom camera pose and dense 3D scene models from monocular endoscopic sequences. Our approach leverages recent advances in self-supervised depth networks to generate pseudo-RGBD frames, then tracks the camera pose using photometric residuals and fuses the registered depth maps in a volumetric representation. We present an extensive experimental evaluation in the public dataset Hamlyn, showing high-quality results and comparisons against relevant baselines. We also release all models and code1 for future comparisons.

关键词： Localization mapping deep learning for visual perception computer vision for medical robotics

来源：评论

学校读者我要写书评

暂无评论

SuperThermal: Matching Thermal as Visible Through Thermal Feature Exploration

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第2期6卷 2690-2697页

作者： Lu, Yawen Lu, Guoyu Rochester Inst Technol Intelligent Vis & Sensing Lab Rochester NY 14586 USA

Infrared Radiation (IR) images that capture the reflected IR signals from the surrounding environment have been applied to pedestrian detection and rescue tasks, especially for thermal images reflecting the surface temperature of the objects at nighttime. However, feature detection, extraction and matching are particularly challenging on thermal images as there are few textures compared to normal RGB images, which brings in an enormous gap in introducing various computer vision tasks such as image matching and retrieval, visual localization and SLAM. In this letter, we propose to design a full pipeline to learn a reliable and robust feature detector and descriptor. Our model trained on cross-spectral stereo and KAIST dataset is able to realize a reliable and accurate matching without further annotations. The proposed method is demonstrated to have superior performance compared with both widely used hand-crafted features and features extracted from the most recent learning-based models on visual and quantitative experiments. It also shows a good generalization and robustness ability on FLIR thermal benchmark even without training or fine-tuning on it.

关键词： Computer vision for transportation deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：