检索结果-内蒙古大学图书馆

Weakly Supervised Disentangled Representation for Goal-Conditioned Reinforcement learning

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第2期7卷 2202-2209页

作者： Qian, Zhifeng You, Mingyu Zhou, Hongjun He, Bin Tongji Univ Coll Elect & Informat Engn Frontiers Sci Ctr Intelligent Autonomous Syst Shanghai 201800 Peoples R China

Goal-conditioned reinforcement learning is a crucial yet challenging algorithm which enables agents to achieve multiple user-specified goals when learning a set of skills in a dynamic environment. However, it typically requires millions of the environmental interactions explored by agents, which is sample-inefficient. In the letter, we propose a skill learning framework DR-GRL that aims to improve the sample efficiency and policy generalization by combining the Disentangled Representation learning and Goal-conditioned visual Reinforcement learning. In a weakly supervised manner, we propose a Spatial Transform AutoEncoder (STAE) to learn an interpretable and controllable representation in which different parts correspond to different object attributes (shape, color, position). Due to the high controllability of the representations, STAE can simply recombine and recode the representations to generate unseen goals for agents to practice themselves. The manifold structure of the learned representation maintains consistency with the physical position, which is beneficial for reward calculation. We empirically demonstrate that DR-GRL significantly outperforms the previous methods in sample efficiency and policy generalization. In addition, DR-GRL is also easy to expand to the real robot.

关键词： deep learning for visual perception reinforcement learning representation learning

来源：评论

学校读者我要写书评

暂无评论

High-Speed Detector for Low-Powered Devices in Aerial Grasping

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2024年第5期9卷 4623-4630页

作者： Kumar, Ashish Behera, Laxmidhar Indian Inst Technol IIT EE Kanpur 208016 India

Autonomous aerial harvesting is a highly complex problem because it requires numerous interdisciplinary algorithms to be executed on mini low-powered computing devices. Object detection is one such algorithm that is compute-hungry. In this context, we make the following contributions: (i) Fast Fruit Detector (FFD), a resource-efficient, single-stage, and postprocessing-free object detector based on our novel latent object representation (LOR) module, query assignment, and prediction strategy. FFD achieves 100 FPS @ FP 32 precision on the latest 10 W NVIDIA Jetson-NX embedded device while co-existing with other time-critical sub-systems such as control, grasping, SLAM, a major achievement of this work, (ii) a method to generate vast amounts of training data without exhaustive manual labelling of fruit images since they consist of a large number of instances, which increases the labelling cost and time, and (iii) an open-source fruit detection dataset having plenty of very small-sized instances that are difficult to detect. Our exhaustive evaluations on our and MinneApple dataset show that FFD, being only a single-scale detector, is more accurate than many representative detectors, e.g. FFD is better than single-scale Faster-RCNN by 10.7 AP, multi-scale Faster-RCNN by 2.3 AP, and better than latest single-scale YOLO-v 8 by 8 AP and multi-scale YOLO-v 8 by 0.3 while being considerably faster.

关键词： Aerial systems: applications agricultural automation deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

UniFuse: Unidirectional Fusion for 360° Panorama Depth Estimation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第2期6卷 1519-1526页

作者： Jiang, Hualie Sheng, Zhe Zhu, Siyu Dong, Zilong Huang, Rui Chinese Univ Hong Kong Shenzhen Inst Artificial Intelligence & Robot Soc Shenzhen 518172 Guangdong Peoples R China Alibaba Cloud AI Lab Hangzhou 311121 Zhejiang Peoples R China

learning depth from spherical panoramas is becoming a popular research topic because a panorama has a full field-of-view of the environment and provides a relatively complete description of a scene. However, applying well-studied CNNs for perspective images to the standard representation of spherical panoramas, i.e., the equirectangular projection, is suboptimal, as it becomes distorted towards the poles. Another representation is the cubemap projection, which is distortion-free but discontinued on edges and limited in the field-of-view. This letter introduces a new framework to fuse features from the two projections, unidirectionally feeding the cubemap features to the equirectangular features only at the decoding stage. Unlike the recent bidirectional fusion approach operating at both the encoding and decoding stages, our fusion scheme is much more efficient. Besides, we also designed a more effective fusion module for our fusion scheme. Experiments verify the effectiveness of our proposed fusion strategy and module, and our model achieves state-of-the-art performance on four popular datasets. Additional experiments show that our model also has the advantages of model complexity and generalization capability.

关键词： deep learning for visual perception omnidirectional vision mapping depth estimation unidirectional fusion

来源：评论

学校读者我要写书评

暂无评论

OSSID: Online Self-Supervised Instance Detection by (And For) Pose Estimation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第2期7卷 3022-3029页

作者： Gu, Qiao Okorn, Brian Held, David Univ Toronto Dept Comp Sci Toronto ON M5S 1A1 Canada Carnegie Mellon Univ Robot Inst Pittsburgh PA 15214 USA

Real-time object pose estimation is necessary for many robot manipulation algorithms. However, state-of-the-art methods for object pose estimation are trained for a specific set of objects;these methods thus need to be retrained to estimate the pose of each new object, often requiring tens of GPU-days of training for optimal performance. In this letter, we propose the OSSID framework, leveraging a slow zero-shot pose estimator to self-supervise the training of a fast detection algorithm. This fast detector can then be used to filter the input to the pose estimator, drastically improving its inference speed. We show that this self-supervised training exceeds the performance of existing zero-shot detection methods on two widely used object pose estimation and detection datasets, without requiring any human annotations. Further, we show that the resulting method for pose estimation has a significantly faster inference speed, due to the ability to filter out large parts of the image. Thus, our method for self-supervised online learning of a detector (trained using pseudo-labels from a slow pose estimator) leads to accurate pose estimation at real-time speeds, without requiring human annotations. Supplementary materials and code can be found at https://***/OSSID/

关键词： Object detection segmentation and categorization deep learning for visual perception RGB-D perception

来源：评论

学校读者我要写书评

暂无评论

What's in the Black Box? The False Negative Mechanisms Inside Object Detectors

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第3期7卷 8510-8517页

作者： Miller, Dimity Moghadam, Peyman Cox, Mark Wildie, Matt Jurdak, Raja CSIRO DATA61 Robot & Autonomous Syst Grp Brisbane Qld 4069 Australia Queensland Univ Technol QUT Trusted Networks Lab Brisbane Qld 4000 Australia

In object detection, false negatives arise when a detector fails to detect a target object. To understand why object detectors produce false negatives, we identify five 'false negative mechanisms,' where each mechanism describes how a specific component inside the detector architecture failed. Focusing on two-stage and one-stage anchor-box object detector architectures, we introduce a framework for quantifying these false negative mechanisms. Using this framework, we investigate why Faster R-CNN and RetinaNet fail to detect objects in benchmark vision datasets and robotics datasets. We show that a detector's false negative mechanisms differ significantly between computer vision benchmark datasets and robotics deployment scenarios. This has implications for the translation of object detectors developed for benchmark datasets to robotics applications.

关键词： deep learning for visual perception object detection segmentation and categorization

来源：评论

学校读者我要写书评

暂无评论

Domain and View-Point Agnostic Hand Action Recognition

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第4期6卷 7823-7830页

作者： Sabater, Alberto Alonso, Inigo Montesano, Luis Murillo, Ana Cristina Univ Zaragoza DIIS I3A Zaragoza 50018 Spain Bitbrain Technol Zaragoza 50008 Spain

Hand action recognition is a special case of action recognition with applications in human-robot interaction, virtual reality or life-logging systems. Building action classifiers able to work for such heterogeneous action domains is very challenging. There are very subtle changes across different actions from a given application but also large variations across domains (e.g. virtual reality vs life-logging). This work introduces a novel skeleton-based hand motion representation model that tackles this problem. The framework we propose is agnostic to the application domain or camera recording view-point. When working on a single domain (intra-domain action classification) our approach performs better or similar to current state-of-the-art methods on well-known hand action recognition benchmarks. And, more importantly, when performing hand action recognition for action domains and camera perspectives which our approach has not been trained for (cross-domain action classification), our proposed framework achieves comparable performance to intra-domain state-of-the-art methods. These experiments show the robustness and generalization capabilities of our framework.

关键词： Human and humanoid motion analysis and synthesis gesture posture and facial expressions deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

Uncertainty-Aware Self-Supervised learning of Spatial perception Tasks

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第4期6卷 6693-6700页

作者： Nava, Mirko Paolillo, Antonio Guzzi, Jerome Gambardella, Luca Maria Giusti, Alessandro USI SUPSI Dalle Molle Inst Artificial Intelligence IDSIA CH-6928 Lugano Switzerland

We propose a general self-supervised learning approach for spatial perception tasks, such as estimating the pose of an object relative to the robot, from onboard sensor readings. The model is learned from training episodes, by relying on: A continuous state estimate, possibly inaccurate and affected by odometry drift;and a detector, that sporadically provides supervision about the target pose. We demonstrate the general approach in three different concrete scenarios: a simulated robot arm that visually estimates the pose of an object of interest;a small differential drive robot using 7 infrared sensors to localize a nearby wall;an omnidirectionalmobile robot that localizes itself in an environment from camera images. Quantitative results show that the approach works well in all three scenarios, and that explicitly accounting for uncertainty yields statistically significant performance improvements. Videos, datasets, and code to reproduce our results are available at: https://***/idsia-robotics/uncertainty-awaressl-spatial- perception.

关键词： deep learning for visual perception deep learning methods

来源：评论

学校读者我要写书评

暂无评论

Cross-View Semantic Segmentation for Sensing Surroundings

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2020年第3期5卷 4867-4873页

作者： Pan, Bowen Sun, Jiankai Leung, Ho Yin Tiga Andonian, Alex Zhou, Bolei MIT Comp Sci & Artificial Intelligence Lab 77 Massachusetts Ave Cambridge MA 02139 USA Chinese Univ Hong Kong Dept Informat Engn Hong Kong Peoples R China

Sensing surroundings plays a crucial role in human spatial perception, as it extracts the spatial configuration of objects as well as the free space from the observations. To facilitate the robot perception with such a surrounding sensing capability, we introduce a novel visual task called Cross-view Semantic Segmentation as well as a framework named View Parsing Network (VPN) to address it. In the cross-view semantic segmentation task, the agent is trained to parse the first-view observations into a top-down-view semantic map indicating the spatial location of all the objects at pixel-level. The main issue of this task is that we lack the real-world annotations of top-down-view data. To mitigate this, we train the VPN in 3D graphics environment and utilize the domain adaptation technique to transfer it to handle real-world data. We evaluate our VPN on both synthetic and real-world agents. The experimental results show that our model can effectively make use of the information from different views and multi-modalities to understanding spatial information. Our further experiment on a LoCoBot robot shows that our model enables the surrounding sensing capability from 2D image input. Code and demo videos can be found at https://***.

关键词： Semantic scene understanding deep learning for visual perception visual learning visual-based navigation computer vision for other robotic applications

来源：评论

学校读者我要写书评

暂无评论

Edge Devices Friendly Self-Supervised Monocular Depth Estimation via Knowledge Distillation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2023年第12期8卷 8470-8477页

作者： Gao, Wei Rao, Di Yang, Yang Chen, Jie Jiangsu Univ Sch Comp Sci & Telecommun Engn Zhenjiang 212013 Peoples R China Northwestern Polytech Univ Sch Marine Sci & Technol CIAIC Xian 710072 Peoples R China

Self-supervised monocular depth estimation (MDE) has great potential for deployment in a wide range of applications, including virtual reality, autonomous driving, and robotics. Nevertheless, most previous studies focused on complex architectures to pursue better performance in MDE. In this letter, we aim to develop a lightweight yet highly effective self-supervised MDE model that can deliver competitive performance in edge devices. We introduce a novel MobileViT-based depth (MViTDepth) network that can effectively capture both local features and global information by leveraging the strengths of convolutional neural networks (CNNs) and a vision transformer (ViT). To further compress the proposed MViTDepth model, we employ knowledge distillation, which leads to improved depth estimation performance. Specifically, the self-supervised MDE MonoViT is used as a teacher model to construct the knowledge distillation loss for optimizing a student model. Experimental results on benchmark datasets demonstrate that the proposed MViTDepth significantly outperforms Monodepth2 in terms of parameters and accuracy, thereby indicating its superiority in application to edge devices.

关键词： deep learning for visual perception autonomous vehicle navigation lightweight monocular depth estimation

来源：评论

学校读者我要写书评

暂无评论

Autonomous Single-Image Drone Exploration With deep Reinforcement learning and Mixed Reality

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第2期7卷 5031-5038页

作者： Devo, Alessandro Mao, Jeffrey Costante, Gabriele Loianno, Giuseppe Univ Perugia Dept Engn I-06125 Perugia Italy NYU Tandon Sch Engn Brooklyn NY 11201 USA

Autonomous exploration is a longstanding goal of the robotics community. Aerial drone navigation has proven to be especially challenging. The stringent requirements on cost, weight, maneuverability, and power consumption do not allow exploration approaches to easily be employed or adapted to different types of environments. End-to-End deep Reinforcement learning (DRL) techniques based on Convolutional Networks approximators, which grant constant-time computation, predefined memory usage, and deliver high visual perception capabilities, represent a very promising alternative to current state of the art solutions relying on metric environment reconstruction. In this work, we address the autonomous exploration problem with aerial robots with a monocular camera based on DRL. Specifically, we propose a novel asymmetric actor-critic model for drone exploration that efficiently leverages ground truth information provided by the simulator environment to speed up learning and enhance final exploration performances. Furthermore, in order to reduce the sim-to-real gap for exploration, we present a novel mixed reality framework that allows an easier, smoother, and safer simulation to real-world transition. Both aspects allow to further exploit the great potential of simulation engines and contribute to reducing the risk associated with directly deploying algorithms on a physical platform with no intermediate step between the simulation and the real world. This is well-known to create several safety concerns and be dangerous when deploying aerial vehicles. Experimental results with a drone exploring multiple environments show the effectiveness of the proposed approach.

关键词： Aerial Systems: Applications reinforcement learning deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：