检索结果-内蒙古大学图书馆

Topological Semantic Mapping by Consolidation of deep visual Features

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第2期7卷 4110-4117页

作者： Sousa, Ygor C. N. Bassani, Hansenclever F. Univ Fed Pernambuco Ctr Informat BR-50740560 Recife PE Brazil

Many works in the recent literature introduce semantic mapping methods that use CNNs (Convolutional Neural Networks) to recognize semantic properties in images. The types of properties (eg.: room size, place category, and objects) and their classes (eg.: kitchen and bathroom, for place category) are usually predefined and restricted to a specific task. Thus, all the visual data acquired and processed during the construction of the maps are lost and only the recognized semantic properties remain on the maps. In contrast, this work introduces a topological semantic mapping method that uses deep visual features extracted by a CNN (GoogLeNet), from 2D images captured in multiple views of the environment as the robot operates, to create, through averages, consolidated representations of the visual features acquired in the regions covered by each topological node. These representations allow flexible recognition of semantic properties of the regions and use in other visual tasks. Experiments with a real-world indoor dataset showed that the method is able to consolidate the visual features of regions and use them to recognize objects and place categories as semantic properties, and to indicate the topological location of images, with very promising results.

关键词： Semantics visualization Feature extraction Image recognition Three-dimensional displays Image segmentation Task analysis Semantic scene understanding deep learning for visual perception representation learning transfer learning topological mapping

来源：评论

学校读者我要写书评

暂无评论

Pose Estimation for Robot Manipulators via Keypoint Optimization and Sim-to-Real Transfer

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第2期7卷 4622-4629页

作者： Lu, Jingpei Richter, Florian Yip, Michael C. Univ Calif San Diego Dept Elect & Comp Engn La Jolla CA 92093 USA

Keypoint detection is an essential budding block !Or many robotic applications like motion capture and pace estimation. Historically, keypoints are detected using uniquely engineered markers such as checkerboards or fiducials. More recently, deep learning methods have been explored as they have the ability to detect user-defined keypoints in a marker-less manner. However, different manually selected keypoints can have uneven performance when it comes to detection and localization. An example of this can be found on symmetric robotic tools where DNN detectors cannot solve the correspondence problem correctly. In this work, we propose a new and autonomous way to define the keypoint locations that overcomes these challenges. The approach involves finding the optimal set of keypoints on robotic manipulators for robust visual detection and localization. Using a robotic simulator as a medium, our algorithm utilizes synthetic data for DNN training, and the proposed algorithm is used to optimize the selection of keypoints through an iterative approach. The results show that when using the optimized keypoints, the detection performance of the DNNs improved significantly. We further use the optimized keypoints for real robotic applications by using domain randomization to bridge the reality gap between the simulator and the physical world. The physical world experiments show how the proposed method can be applied to the wide-breadth of robotic applications that require visual feedback, such as camera-to-robot calibration, robotic tool tracking, and end-effector pose estimation. As a way to encourage further research in this topic, we establish the "Robot Pose" dataset, comprising calibration and tracking problems and ground truth data, available online(1).

关键词： visual tracking deep learning for visual perception perception for grasping and manipulation

来源：评论

学校读者我要写书评

暂无评论

Semi-Supervised learning: Structure, Reflectance and Lighting Estimation From a Night Image Pair

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第2期7卷 976-983页

作者： Wang, Ke Shen, Shaojie Hong Kong Univ Sci & Technol Dept Elect & Comp Engn Hong Kong Peoples R China

While unsupervised approaches have been proposed to estimate reflectance and shading layers for images, the decomposition process is a challenging, under-determined inverse problem. Previous unsupervised approaches for intrinsic decomposition of images are strictly reliant on special image sequences, for example, image sequences for a fixed scene with changing illumination or multi-view images containing rich illumination variation. As an alternative to these unsupervised intrinsic decomposition methods, we propose a semi-supervised method. Our method adopts a simplified scene representation to greatly reduce the complexity of spatially varying lighting, which allows us to partially restore the lighting from a night image pair. Moreover, we design several novel unsupervised guiding losses, and the training data used in our method are easily collected by setting different exposure times for a standard stereo setup. We further demonstrate the effectiveness of the proposed method by quantitatively and qualitatively comparing our method with recent works on the modified MPI dataset and a collected highly dynamic dataset named NightCampus. The NightCampus dataset has been released at https://***/ kewangtt/srlefnip.

关键词： deep learning for visual perception intrinsic image decomposition and lighting estimation visual learning

来源：评论

学校读者我要写书评

暂无评论

A Continuous Robot Vision Approach for Predicting Shapes and visually Perceived Weights of Garments

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第3期7卷 7950-7957页

作者： Duan, Li Aragon-Camarasa, Gerardo Univ Glasgow Sch Comp Sci Glasgow G12 8RZ Lanark Scotland

We present a continuous perception approach that learns geometric and physical similarities between garments by continuously observing a garment while a robot picks it up from a table. The aim is to capture and encode geometric and physical characteristics of a garment into a manifold where a decision can be carried out, such as predicting the garment's shape class and its visually perceived weight. Our approach features an early stop strategy, which means that a robot does not need to observe a full video sequence of a garment being picked up from a crumpled to a hanging state to make a prediction, taking 8 seconds in average to classify garment shapes. In our experiments, we find that our approach achieves prediction accuracies of 93% for shape classification and 98.5% for predicting weights and advances state-of-art approaches in similar robotic perception tasks by 22%for shape classification.

关键词： Computer vision for automation deep learning for visual perception visual learning AI-enabled robotics

来源：评论

学校读者我要写书评

暂无评论

End-to-End Optimization of LiDAR Beam Configuration for 3D Object Detection and Localization

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第2期7卷 2242-2249页

作者： Voedisch, Niclas Unal, Ozan Li, Ke Van Gool, Luc Dai, Dengxin Swiss Fed Inst Technol Comp Vis Lab CH-8092 Zurich Switzerland Univ Freiburg Autonomous Intelligent Syst D-79110 Freiburg Germany Katholieke Univ Leuven Proc Speech & Images B-3001 Leuven Belgium MPI Informat Vis Autonomous Syst D-66123 Saarbrucken Germany

Existing learning methods for LiDAR-based applications use 3D points scanned under a pre-determined beam configuration, e.g., the elevation angles of beams are often evenly distributed. Those fixed configurations are task-agnostic, so simply using them can lead to sub-optimal performance. In this work, we take a new route to learn to optimize the LiDAR beam configuration for a given application. Specifically, we propose a reinforcement learning-based learning-to-optimize (RL-L2O) framework to automatically optimize the beam configuration in an end-to-end manner for different LiDAR-based applications. The optimization is guided by the final performance of the target task and thus our method can be integrated easily with any LiDAR-based application as a simple drop-in module. The method is especially useful when a low-resolution (low-cost) LiDAR is needed, for instance, for system deployment at a massive scale. We use our method to search for the beam configuration of a low-resolution LiDAR for two important tasks: 3D object detection and localization. Experiments show that the proposed RL-L2O method improves the performance in both tasks significantly compared to the baseline methods. We believe that a combination of our method with the recent advances of programmable LiDARs can start a new research direction for LiDAR-based active perception.

关键词： Reinforcement learning deep learning for visual perception localization adaptive LiDAR

来源：评论

学校读者我要写书评

暂无评论

Randomized-to-Canonical Model Predictive Control for Real-World visual Robotic Manipulation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第4期7卷 8964-8971页

作者： Yamanokuchi, Tomoya Kwon, Yuhwan Tsurumine, Yoshihisa Uchibe, Eiji Morimoto, Jun Matsubara, Takamitsu Nara Inst Sci & Technol NAIST Grad Sch Informat Sci Nara 6300192 Japan Adv Telecommun Res Inst Int ATR Kyoto 6190288 Japan Kyoto Univ Dept Syst Sci Grad Sch Informat Kyoto 6068501 Japan

Many works have recently explored Sim-to-real transferable visual model predictive control (MPC). However, such works are limited to one-shot transfer, where real-world data must be collected once to perform the sim-to-real transfer, which remains a significant human effort in transferring the models learned in simulations to new domains in the real world. To alleviate this problem, we first propose a novel model-learning framework called Kalman Randomized-to-Canonical Model (KRC-model). This framework is capable of extracting task-relevant intrinsic features and their dynamics from randomized images. We then propose Kalman Randomized-to-Canonical Model Predictive Control (KRC-MPC) as a zero-shot sim-to-real transferable visual MPC using KRC-model. The effectiveness of our method is evaluated through a valve rotation task by a robot hand in both simulation and the real world, and a block mating task in simulation. The experimental results show that KRC-MPC can he applied to various real domains and tasks in a zero-shot manner.

关键词： deep learning for visual perception machine learning for robot control model learning for control

来源：评论

学校读者我要写书评

暂无评论

Navigation-Oriented Scene Understanding for Robotic Autonomy: learning to Segment Driveability in Egocentric Images

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第2期7卷 2913-2920页

作者： Humblot-Renaux, Galadrielle Marchegiani, Letizia Moeslund, Thomas B. Gade, Rikke Aalborg Univ Visual Anal & Percept Lab DK-9220 Aalborg Denmark Aalborg Univ Dept Elect Syst DK-9220 Aalborg Denmark

This work tackles scene understanding for outdoor robotic navigation, solely relying on images captured by an on-board camera. Conventional visual scene understanding interprets the environment based on specific descriptive categories. However, such a representation is not directly interpretable for decision-making and constrains robot operation to a specific domain. Thus, we propose to segment egocentric images directly in terms of how a robot can navigate in them, and tailor the learning problem to an autonomous navigation task. Building around an image segmentation network, we present a generic affordance consisting of 3 driveability levels which can broadly apply to both urban and off-road scenes. By encoding these levels with soft ordinal labels, we incorporate inter-class distances during learning which improves segmentation compared to standard "hard" one-hot labelling. In addition, we propose a navigation-oriented pixel-wise loss weighting method which assigns higher importance to safety-critical areas. We evaluate our approach on large-scale public image segmentation datasets ranging from sunny city streets to snowy forest trails. In a cross-dataset generalization experiment, we show that our affordance learning scheme can be applied across a diverse mix of datasets and improves driveability estimation in unseen environments compared to general-purpose, single-dataset segmentation.

关键词： Computer vision for transportation deep learning for visual perception semantic scene understanding

来源：评论

学校读者我要写书评

暂无评论

REVE-CE: Remote Embodied visual Referring Expression in Continuous Environment

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第2期7卷 1494-1501页

作者： Li, Xinghang Guo, Di Liu, Huaping Sun, Fuchun Tsinghua Univ Beijing Natl Res Ctr Informat Sci & Technol Inst Artificial Intelligence Beijing 100084 Peoples R China Tsinghua Univ Dept Comp Sci & Technol Beijing 100084 Peoples R China

It has always been a great challenge for the robot to navigate in the visual world following natural language instructions. Recently, several tasks such as the Vision-and-Language Navigation (VLN) and Remote Embodied visual Referring Expression in Real Indoor Environments (REVERIE) are proposed trying to solve this challenge. And the most significant difference between VLN and REVERIE tasks is that REVERIE uses a higher guidance level instruction. However, the navigation process of REVERIE is implemented in a discrete environment, which is unrealistic in real world scenarios. To make the REVERIE task more consistent with the real physical world, we develop a new task of Remote Embodied visual Referring Expression in Continuous Environment, namely REVE-CE, in which the agent executes a much longer sequence of low-level actions given language instructions. Furthermore, we propose a multi-branch cross modal attention (MBCMA) framework to solve the proposed REVE-CE task. Extensive experiments are conducted demonstrating that the proposed framework greatly outperforms the state-of-the-art VLN baselines and a new benchmark for the proposed REVE-CE task is built.

关键词： deep learning for visual perception embodied cognitive science perception-action coupling vision-based navigation

来源：评论

学校读者我要写书评

暂无评论

SLFNet: A Stereo and LiDAR Fusion Network for Depth Completion

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第4期7卷 10605-10612页

作者： Zhang, Yongjian Wang, Longguang Li, Kunhong Fu, Zhiheng Guo, Yulan Sun Yat Sen Univ Sch Elect & Commun Engn Shenzhen Campus Shenzhen 518107 Peoples R China Natl Univ Def Technol Coll Elect Sci & Technol Changsha 410073 Peoples R China Univ Western Australia UWA Dept Comp Sci & Software Engn Perth WA 6009 Australia

Acquiring dense and precise depth information in real time is highly demanded for robotic perception and automatic driving. Motivated by the complementary nature of stereo images and LiDAR point clouds, we propose an efficient stereo-LiDAR fusion network (SLFNet) to predict a dense depth map of a scene. Specifically, the LiDAR point cloud is first projected onto each image plane of the stereo images to generate sparse RGB-D maps. Then, multi-modal feature fusion is performed between RGB image and sparse RGB-D map of the same viewpoint, and the resultant features are utilized to generate a coarse disparity map for stereo fusion. Next, complementary geometric information in stereo images and sparse RGB-D maps are incorporated to perform occlusion-aware refinement. Finally, an edge-aware refinement module is conducted to encourage the depth discontinuities to be consistent with edges in the image. Experimental results demonstrate that our network can effectively fuse the stereo images and point clouds to produce accurate depth estimations at 6 FPS, which is 8x faster than existing methods. Comparative results show that our network achieves the state-of-the-art performance on the KITTI and Virtual KITTI2 datasets.

关键词： deep learning for visual perception sensor fusion

来源：评论

学校读者我要写书评

暂无评论

Bird's-Eye-View Panoptic Segmentation Using Monocular Frontal View Images

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第2期7卷 1968-1975页

作者： Gosala, Nikhil Valada, Abhinav Univ Freiburg Dept Comp Sci Freiburg Germany

Bird's-Eye-View (BEV) maps have emerged as one of the most powerful representations for scene understanding due to their ability to provide rich spatial context while being easy to interpret and process. Such maps have found use in many real-world tasks that extensively rely on accurate scene segmentation as well as object instance identification in the BEV space for their operation. However, existing segmentation algorithms only predict the semantics in the BEV space, which limits their use in applications where the notion of object instances is also critical. In this work, we present the first BEV panoptic segmentation approach for directly predicting dense panoptic segmentation maps in the BEV, given a single monocular image in the frontal view (FV). Our architecture follows the top-down paradigm and incorporates a novel dense transformer module consisting of two distinct transformers that learn to independently map vertical and flat regions in the input image from the FVto the BEV. Additionally, we derive a mathematical formulation for the sensitivity of the FV-BEV transformation which allows us to intelligently weight pixels in the BEV space to account for the varying descriptiveness across the FV image. Extensive evaluations on the KITTI-360 and nuScenes datasets demonstrate that our approach exceeds the state-of-the-art in the PQ metric by 3.61 pp and 4.93 pp respectively.

关键词： Semantic scene understanding object detection segmentation and categorization deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：