检索结果-内蒙古大学图书馆

Self-Supervised learning of Domain-Invariant Local Features for Robust visual Localization Under Challenging Conditions

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第2期6卷 2753-2760页

作者： Venator, Moritz El Himer, Yassine Aklanoglu, Selcuk Bruns, Erich Maier, Andreas Volkswagen Car SW Org D-85057 Ingolstadt Germany Friedrich Alexander Univ Erlangen Nurnberg Pattern Recognit Lab D-91058 Erlangen Germany Tech Univ Munich D-80333 Munich Germany

visual localization provides the basis for many robotics applications such as autonomous navigation or augmented reality. Especially in outdoor scenes, robust localization requires local features which can be reliably extracted and matched under changing conditions. Previous approaches have applied generative image-to-image translation models to align images in a single domain before correspondence search. In this letter, we invert the concept and elaborate why it is more promising to use image domain adaptation for training of robust local features. Integrating this idea into a self-supervised training framework, we show in various experiments covering image matching, visual localization, and scene reconstruction that our Domain-Invariant SuperPoint (DISP) outperforms existing self-supervised methods in terms of repeatability, generalization, and robustness. In contrast to competitive supervised local features, our modular and fully self-supervised approach can be easily adapted to different domains and localization tasks as it does not require ground truth correspondences for training.

关键词： Computer vision for transportation deep learning for visual perception localization

来源：评论

学校读者我要写书评

暂无评论

GEM: Glare or Gloom, I Can Still See You - End-to-End Multi-Modal Object Detection

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第4期6卷 6321-6328页

作者： Mazhar, Osama Babuska, Robert Kober, Jens Delft Univ Technol Cognit Robot Dept NL-2628 CD Delft Netherlands Czech Tech Univ Czech Inst Informat Robot & Cybernet Prague 16636 Czech Republic

deep neural networks designed for vision tasks are often prone to failure when they encounter environmental conditions not covered by the training data. Single-modal strategies are insufficient when the sensor fails to acquire information due to malfunction or its design limitations. Multi-sensor configurations are known to provide redundancy, increase reliability, and are crucial in achieving robustness against asymmetric sensor failures. To address the issue of changing lighting conditions and asymmetric sensor degradation in object detection, we develop a multi-modal 2D object detector, and propose deterministic and stochastic sensor-aware feature fusion strategies. The proposed fusion mechanisms are driven by the estimated sensor measurement reliability values/weights. Reliable object detection in harsh lighting conditions is essential for applications such as self-driving vehicles and human-robot interaction. We also propose a new "r-blended" hybrid depth modality for RGB-D sensors. Through extensive experimentation, we show that the proposed strategies outperform the existing state-of-the-art methods on the FLIR-Thermal dataset, and obtain promising results on the SUNRGB-D dataset. We additionally record a new RGB-Infra indoor dataset, namely L515-Indoors, and demonstrate that the proposed object detection methodologies are highly effective for a variety of lighting conditions.

关键词： computer vision for automation deep learning for visual perception object detection RGB-D perception segmentation and categorization sensor fusion

来源：评论

学校读者我要写书评

暂无评论

Cross-Modal Representation learning for Lightweight and Accurate Facial Action Unit Detection

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第4期6卷 7619-7626页

作者： Chen, Yingjie Wu, Han Wang, Tao Wang, Yizhou Liang, Yun Peking Univ Dept Elect Engn & Comp Sci Beijing 100871 Peoples R China

In this letter, we focus on designing an effective method for lightweight and accurate facial action unit (AU) detection, which is essential for emotional communication in most human-robot interaction scenarios. AU detection is a delicate and challenging task because the subtle fleeting appearance changes caused by AUs are very difficult to catch and express. Therefore existing approaches mainly deal with static facial states or frame-level temporal relationships. The dynamic process of facial muscle movement, as the core feature of AU, is yet ignored and rarely exploited by prior studies. Based on such observation, we propose Flow Supervised Module (FSM) to explicitly capture the dynamic facial movement in the form of Flow and use the learned Flow to provide supervision signals for the detection model during the training stage effectively and efficiently. Furthermore, the proposed FSM can be easily incorporated into various backbone networks and boost their performance. Extensive experiments are conducted on two benchmark datasets, DISFA and BP4D, showing state-of-the-art performance with competitive detection speed.

关键词： Gold Feature extraction Optical imaging visualization Task analysis Optical computing deep learning Gesture posture and facial expressions deep learning for visual perception emotional robotics

来源：评论

学校读者我要写书评

暂无评论

Pose Consistency KKT-Loss for Weakly Supervised learning of Robot-Terrain Interaction Model

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第3期6卷 5477-5484页

作者： Salansky, Vojtech Zimmermann, Karel Petricek, Tomas Svoboda, Tomas Czech Tech Univ Fac Elect Engn Dept Cybernet Prague 16636 Czech Republic

We address the problem of self-supervised learning for predicting the shape of supporting terrain (i.e. the terrain which will provide rigid support for the robot during its traversal) from sparse input measurements. The learning method exploits two types of ground-truth labels: dense 2.5D maps and robot poses, both estimated by a usual SLAM procedure from offline recorded measurements. We show that robot poses are required because straightforward supervised learning from the 3D maps only suffers from: (i) exaggerated height of the supporting terrain caused by terrain flexibility (vegetation, shallow water, snow or sand) and (ii) missing or noisy measurements caused by high spectral absorbance or non-Lambertian reflectance of the measured surface. We address the learning from robot poses by introducing a novel KKT-loss, which emerges as the distance from necessary Karush-Kuhn-Tucker conditions for constrained local optima of a simplified first-principle model of the robot-terrain interaction. We experimentally verify that the proposed weakly supervised learning from ground-truth robot poses boosts the accuracy of predicted support heightmaps and increases the accuracy of estimated robot poses. All experiments are conducted on a dataset captured by a real platform. Both the dataset and codes which replicates experiments in the paper are made publicly available as a part of the submission.

关键词： deep learning for visual perception representation learning

来源：评论

学校读者我要写书评

暂无评论

PlaceNet: A multi-scale semantic-aware model for visual loop closure detection

引用

ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE 2023年第1期119卷

作者： Osman, Hussein Darwish, Nevin Bayoumi, AbdElMoniem Cairo Univ Fac Engn Dept Comp Engn Gamaa St Giza 12613 Egypt Ecole Polytech Fed Lausanne EPFL INN 134 Batiment INNStn 14 CH-1015 Lausanne Switzerland

Loop closure detection helps simultaneous localization and mapping systems reduce map and state uncertainty via recognizing previously visited places along the path of a mobile robot. However, visual loop closure detection is susceptible to scenes with dynamic objects and changes in illumination, background, and weather conditions. This paper introduces PlaceNet, a novel plug-and-play model for visual loop closure detection. PlaceNet is a multi-scale deep autoencoder network augmented with a semantic fusion layer for scene understanding. The main idea of PlaceNet is to learn where not to look in a dynamic scene full of moving objects, i.e., avoid being distracted by dynamic objects to focus on the scene landmarks instead. We train PlaceNet to identify dynamic objects in scenes via learning a grayscale semantic map indicating the position of static and moving objects in the image. PlaceNet generates semantic-aware deep features that are robust to dynamic environments and scale invariant. We evaluated our method on different challenging indoor and outdoor benchmarks. To conclude, PlaceNet demonstrated competitive results compared to the state-of-the-art methods over various datasets used in our experiments.

关键词： visual loop closure detection deep learning for visual perception visual SLAM Localization

来源：评论

学校读者我要写书评

暂无评论

Adversarial Attacks on Camera-LiDAR Models for 3D Car Detection

Adversarial Attacks on Camera-LiDAR Models for 3D Car Detect...

引用

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

作者： Abdelfattah, Mazen Yuan, Kaiwen Wang, Z. Jane Ward, Rabab Univ British Columbia Dept Elect & Comp Engn Vancouver BC Canada

ISBN: (纸本)9781665417143

Most autonomous vehicles (AVs) rely on LiDAR and RGB camera sensors for perception. Using these point cloud and image data, perception models based on deep neural nets (DNNs) have achieved state-of-the-art performance in 3D detection. The vulnerability of DNNs to adversarial attacks have been heavily investigated in the RGB image domain and more recently in the point cloud domain, but rarely in both domains simultaneously. Multi-modal perception systems used in AVs can be divided into two broad types: cascaded models which use each modality independently, and fusion models which learn from different modalities simultaneously. We propose a universal and physically realizable adversarial attack for each type, and study and contrast their respective vulnerabilities to attacks. We place a single adversarial object with specific shape and texture on top of a car with the objective of making this car evade detection. Evaluating on the popular KITTI benchmark, our adversarial object made the host vehicle escape detection by each model type more than 50% of the time. The dense RGB input contributed more to the success of the adversarial attacks on both cascaded and fusion models.

关键词： RGB-D perception sensor fusion deep learning for visual perception adversarial attacks 3D detection

来源：评论

学校读者我要写书评

暂无评论

Future Frame Prediction for Robot-Assisted Surgery 27th

Future Frame Prediction for Robot-Assisted Surgery

引用

27th International Conference on Information Processing in Medical Imaging (IPMI)

作者： Gao, Xiaojie Jin, Yueming Zhao, Zixu Dou, Qi Heng, Pheng-Ann Chinese Univ Hong Kong Dept Comp Sci & Engn Hong Kong Peoples R China CUHK T Stone Robot Inst Hong Kong Peoples R China

ISBN: (纸本)9783030781903;9783030781910

Predicting future frames for robotic surgical video is an interesting, important yet extremely challenging problem, given that the operative tasks may have complex dynamics. Existing approaches on future prediction of natural videos were based on either deterministic models or stochastic models, including deep recurrent neural networks, optical flow, and latent space modeling. However, the potential in predicting meaningful movements of robots with dual arms in surgical scenarios has not been tapped so far, which is typically more challenging than forecasting independent motions of one arm robots in natural scenarios. In this paper, we propose a ternary prior guided variational autoencoder (TPG-VAE) model for future frame prediction in robotic surgical video sequences. Besides content distribution, our model learns motion distribution, which is novel to handle the small movements of surgical tools. Furthermore, we add the invariant prior information from the gesture class into the generation process to constrain the latent space of our model. To our best knowledge, this is the first time that the future frames of dual arm robots are predicted considering their unique characteristics relative to general robotic videos. Experiments demonstrate that our model gains more stable and realistic future frame prediction scenes with the suturing task on the public JIGSAWS dataset.

关键词： Video prediction for medical robotics deep learning for visual perception Medical robots and systems

来源：评论

学校读者我要写书评

暂无评论

Towards Better Surgical Instrument Segmentation in Endoscopic Vision: Multi-Angle Feature Aggregation and Contour Supervision

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2020年第4期5卷 6639-6646页

作者： Qin, Fangbo Lin, Shan Li, Yangming Bly, Randall A. Moe, Kris S. Hannaford, Blake Chinese Acad Sci Inst Automat Res Ctr Precis Sensing & Control Beijing 100190 Peoples R China Univ Washington UW Dept Elect Engn Seattle WA 98195 USA Univ Washington Dept Otolaryngol Head & Neck Surg Seattle WA 98195 USA Rochester Inst Technol Dept Elect Comp & Telecommun Engn Technol Rochester NY 14623 USA

Accurate and real-time surgical instrument segmentation is important in the endoscopic vision of robot-assisted surgery, and significant challenges are posed by frequent instrument-tissue contacts and continuous change of observation perspective. For these challenging tasks more and more deep neural networks (DNN) models are designed in recent years. We are motivated to propose a general embeddable approach to improve these current DNN segmentation models without increasing the model parameter number. Firstly, observing the limited rotation-invariance performance of DNN, we proposed the Multi-Angle Feature Aggregation (MAFA) method, leveraging active image rotation to gain richer visual cues and make the prediction more robust to instrument orientation changes. Secondly, in the end-to-end training stage, the auxiliary contour supervision is utilized to guide the model to learn the boundary awareness, so that the contour shape of segmentation mask is more precise. The proposed method is validated with ablation experiments on the novel Sinus-Surgery datasets collected from surgeons' operations, and is compared to the existing methods on a public dataset collected with a da Vinci Xi Robot.

关键词： Computer vision for medical robotics medical robots and systems deep learning for visual perception object detection segmentation and categorization

来源：评论

学校读者我要写书评

暂无评论

AMAE: Adaptive Motion-Agnostic Encoder for Event-Based Object Classification

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2020年第3期5卷 4596-4603页

作者： Deng, Yongjian Li, Youfu Chen, Hao City Univ Hong Kong Dept Mech Engn Hong Kong Peoples R China Nanyang Technol Univ Sch Comp Sci & Engn Singapore 639798 Singapore

Event cameras, with low power consumption, high temporal resolution, and high dynamic range, have been used increasingly in computer vision. These superior characteristics enable event cameras to perform low-energy and high-response object classification tasks in challenging scenarios. Nevertheless, specific encoding methods for event-based classification are required owing to the unconventional output of event cameras. Existing event-based encoding methods have focused on extracting semantic and motion information in event signals. However, two main problems exist in these methods: (i) the motion information of event signals leads to mispredictions by the classifiers. (ii) effective evaluation methods to validate the motion robustness of event-based classification models have yet to be proposed. In this work, we introduce an adaptive motion-agnostic encoder for event streams to address the first problem. The proposed encoder would allow us to extract indistinguishable semantic information from an object in different motion circumstances. In addition, we propose a novel motion inconsistency evaluation method to assess the motion robustness of the classification models. We apply our method to several benchmark datasets and evaluate it using motion consistency and inconsistency testing methods. Classification performance shows that our proposed encoder outperforms state-of-the-art methods by a large margin.

关键词： Feature extraction Semantics Cameras Task analysis Encoding Data mining Robustness Object detection segmentation and categor-ization recognition deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

Simultaneously learning Corrections and Error Models for Geometry-Based visual Odometry Methods

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2020年第4期5卷 6536-6543页

作者： De Maio, Andrea Lacroix, Simon Univ Toulouse LAAS CNRS F-31031 Toulouse France

This letter fosters the idea that deep learning methods can be used to complement classical visual odometry pipelines to improve their accuracy and to associate uncertainty models to their estimations. We show that the biases inherent to the visual odometry process can be faithfully learned and compensated for, and that a learning architecture associated with a probabilistic loss function can jointly estimate a full covariance matrix of the residual errors, defining an error model capturing the heteroscedasticity of the process. Experiments on autonomous driving image sequences assess the possibility to concurrently improve visual odometry and estimate an error associated with its outputs.

关键词： Covariance matrices Uncertainty visual odometry Estimation Matrix decomposition Probabilistic logic Motion estimation deep learning for visual perception visual-based navigation localization

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：