检索结果-内蒙古大学图书馆

Learning Deep Sensorimotor Policies for vision-Based Autonomous Drone Racing

学校读者我要写书评

暂无评论

Learning Deep Sensorimotor Policies for Vision-Based Autonom...

IEEE/RSJ International Conference on Intelligent Robots and systems (IROS)

作者： Jiawei Fu Yunlong Song Yan Wu Fisher Yu Davide Scaramuzza Department of Informatics Robotics and Perception Group University of Zurich Switzerland Department of Neuroinformatics University of Zurich and ETH Zurich Switzerland Visual Intelligence and Systems Group in the Computer Vision Lab ETH Zurich

The development of effective vision-based algorithms has been a significant challenge in achieving autonomous drones, which promise to offer immense potential for many real-world applications. This paper investigates learning deep sensorimotor policies for vision-based drone racing, which is a particularly demanding setting for testing the limits of an algorithm. Our method combines feature representation learning to extract task-relevant feature representations from high-dimensional image inputs with a learning-by-cheating framework to train a deep sensorimotor policy for vision-based drone racing. This approach eliminates the need for globally-consistent state estimation, trajectory planning, and handcrafted control design, allowing the policy to directly infer control commands from raw images, similar to human pilots. We conduct experiments using a realistic simulator and show that our vision-based policy can achieve state-of-the-art racing performance while being robust against unseen visual disturbances. Our study suggests that consistent feature embeddings are essential for achieving robust control performance in the presence of visual disturbances. The key to acquiring consistent feature embeddings is utilizing contrastive learning along with data augmentation. Video: https://***/AX_fcnW9yqE

关键词：

Renovating Names in Open-Vocabulary Segmentation Benchmarks 38

学校读者我要写书评

暂无评论

Renovating Names in Open-Vocabulary Segmentation Benchmarks

38th Conference on Neural Information Processing systems, NeurIPS 2024

作者： Huang, Haiwen Peng, Songyou Zhang, Dan Geiger, Andreas Bosch IoC Lab University of Tübingen Germany Tübingen AI Center Germany Autonomous Vision Group University of Tübingen Germany ETH Zurich Switzerland MPI for Intelligent Systems Tübingen Germany Bosch Center for Artificial Intelligence Germany

Names are essential to both human cognition and vision-language models. Open-vocabulary models utilize class names as text prompts to generalize to categories unseen during training. However, the precision of these names is often overlooked in existing datasets. In this paper, we address this underexplored problem by presenting a framework for "renovating" names in open-vocabulary segmentation benchmarks (RENOVATE). Our framework features a renaming model that enhances the quality of names for each visual segment. Through experiments, we demonstrate that our renovated names help train stronger open-vocabulary models with up to 15% relative improvement and significantly enhance training efficiency with improved data quality. We also show that our renovated names improve evaluation by better measuring misclassification and enabling fine-grained model analysis. We provide our code and relabelings for several popular segmentation datasets to the research community on our project page: https://***/renovate/. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

Learning Deep Sensorimotor Policies for vision-based Autonomous Drone Racing

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Fu, Jiawei Song, Yunlong Wu, Yan Yu, Fisher Scaramuzza, Davide Robotics and Perception Group Department of Informatics University of Zurich Department of Neuroinformatics University of Zurich ETH Zurich Switzerland Visual Intelligence and Systems Group in the Computer Vision Lab at ETH Zurich Switzerland

unstructured environments, enabling various real-world applications. However, the lack of effective vision-based algorithms has been a stumbling block to achieving this goal. Existing systems often require hand-engineered components for state estimation, planning, and control. Such a sequential design involves laborious tuning, human heuristics, and compounding delays and errors. This paper tackles the vision-based autonomous-drone-racing problem by learning deep sensorimotor policies. We use contrastive learning to extract robust feature representations from the input images and leverage a two-stage learning-by-cheating framework for training a neural network policy. The resulting policy directly infers control commands with feature representations learned from raw images, forgoing the need for globally-consistent state estimation, trajectory planning, and handcrafted control design. Our experimental results indicate that our vision-based policy can achieve the same level of racing performance as the state-based policy while being robust against different visual disturbances and distractors. We believe this work serves as a stepping-stone toward developing intelligent vision-based autonomous systems that control the drone purely from image inputs, like human pilots. Copyright © 2022, The Authors. All rights reserved.

关键词： Drones

Learning-based Relational Object Matching Across Views

学校读者我要写书评

暂无评论

Learning-based Relational Object Matching Across Views

IEEE International Conference on Robotics and Automation (ICRA)

作者： Cathrin Elich Iro Armeni Martin R. Oswald Marc Pollefeys Joerg Stueckler Embodied Vision Group Max Planck Institute for Intelligent Systems Tuebingen Germany Max Planck ETH Center for Learning Systems Computer Vision and Geometry Lab ETH Zurich Switzerland University of Amsterdam Netherlands Microsoft Mixed Reality and AI Lab Zurich Switzerland

Intelligent robots require object-level scene understanding to reason about possible tasks and interactions with the environment. Moreover, many perception tasks such as scene reconstruction, image retrieval, or place recognition can benefit from reasoning on the level of objects. While keypoint-based matching can yield strong results for finding correspondences for images with small to medium view point changes, for large view point changes, matching semantically on the object-level becomes advantageous. In this paper, we propose a learning-based approach which combines local keypoints with novel object-level features for matching object detections between RGB images. We train our object-level matching features based on appearance and inter-frame and cross-frame spatial relations between objects in an associative graph neural network. We demonstrate our approach in a large variety of views on realistically rendered synthetic images. Our approach compares favorably to previous state-of-the-art object-level matching approaches and achieves improved performance over a pure keypoint-based approach for large view-point changes.

关键词：

Learning-based Relational Object Matching Across Views

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Elich, Cathrin Armeni, Iro Oswald, Martin R. Pollefeys, Marc Stueckler, Joerg Embodied Vision Group Max Planck Institute for Intelligent Systems Tuebingen Germany The Max Planck ETH Center for Learning Systems The Computer Vision and Geometry Lab ETH Zurich Switzerland University of Amsterdam Netherlands Microsoft Mixed Reality and AI Lab Zurich Switzerland

关键词： Intelligent robots

Renovating Names in Open-Vocabulary Segmentation Benchmarks

学校读者我要写书评

暂无评论

arXiv 2024年

Names are essential to both human cognition and vision-language models. Open-vocabulary models utilize class names as text prompts to generalize to categories unseen during training. However, the precision of these names is often overlooked in existing datasets. In this paper, we address this underexplored problem by presenting a framework for "renovating" names in open-vocabulary segmentation benchmarks (RENOVATE). Our framework features a renaming model that enhances the quality of names for each visual segment. Through experiments, we demonstrate that our renovated names help train stronger open-vocabulary models with up to 15% relative improvement and significantly enhance training efficiency with improved data quality. We also show that our renovated names improve evaluation by better measuring misclassification and enabling fine-grained model analysis. We will provide our code and relabelings for several popular segmentation datasets (MS COCO, ADE20K, Cityscapes) to the research community. © 2024, CC BY.

关键词：

LocalViT: Analyzing Locality in vision Transformers

学校读者我要写书评

暂无评论

LocalViT: Analyzing Locality in Vision Transformers

IEEE/RSJ International Conference on Intelligent Robots and systems (IROS)

作者： Yawei Li Kai Zhang Jiezhang Cao Radu Timofte Michele Magno Luca Benini Luc Van Goo Computer Vision Lab D-ITET ETH Zurich Switzerland Center for Artificial Intelligence and Data Science (CAIDAS) University of Wurzburg Germany Center for Project-Based Learning D-ITET ETH Zurich Switzerland Integrated Systems Laboratory D-ITET ETH Zurich Switzerland Department of Electrical Electronic and Information Engineering University of Bologna Italy Processing Speech and Images (PSI) KU Leuven Belgium

The aim of this paper is to study the influence of locality mechanisms in vision transformers. Transformers originated from machine translation and are particularly good at modelling long-range dependencies within a long sequence. Although the global interaction between the token embeddings could be well modelled by the self-attention mechanism of transformers, what is lacking is a locality mechanism for infor-mation exchange within a local region. In this paper, locality mechanism is systematically investigated by carefully designed controlled experiments. We add locality to vision transformers into the feed-forward network. This seemingly simple solution is inspired by the comparison between feed-forward networks and inverted residual blocks. The importance of locality mechanisms is validated in two ways: 1) A wide range of design choices (activation function, layer placement, expansion ratio) are available for incorporating locality mechanisms and proper choices can lead to a performance gain over the baseline, and 2) The same locality mechanism is successfully applied to vision transformers with different architecture designs, which shows the generalization of the locality concept. For ImageNet2012 classification, the locality-enhanced transformers outperform the baselines Swin-T [1], DeiT-T [2] and PVT-T [3] by 1.0%, 2.6 % and 3.1 % with a negligible increase in the number of parameters and computational effort. Code is available at https://***/ofsoundof/LocalViT.

关键词：

A Unified Framework for Event-based Frame Interpolation with Ad-hoc Deblurring in the Wild

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Sun, Lei Gehrig, Daniel Sakaridis, Christos Gehrig, Mathias Liang, Jingyun Sun, Peng Xu, Zhijie Wang, Kaiwei Van Gool, Luc Scaramuzza, Davide National Research Center for Optical Instrumentation Zhejiang University Hangzhou310027 China Robotics and Perception Group University of Zurich Zurich8050 Switzerland Computer Vision Lab ETH Zurich Zurich8092 Switzerland Centre for Visual and Immersive Computing Huddersfield University HD1 3DH United Kingdom INSAIT Sofia University "St. Kliment Ohridski" Bulgaria

Effective video frame interpolation hinges on the adept handling of motion in the input scene. Prior work acknowledges asynchronous event information for this, but often overlooks whether motion induces blur in the video, limiting its scope to sharp frame interpolation. We instead propose a unified framework for event-based frame interpolation that performs deblurring ad-hoc and thus works both on sharp and blurry input videos. Our model consists in a bidirectional recurrent network that incorporates the temporal dimension of interpolation and fuses information from the input frames and the events adaptively based on their temporal proximity. To enhance the generalization from synthetic data to real event cameras, we integrate self-supervised framework with the proposed model to enhance the generalization on real-world datasets in the wild. At the dataset level, we introduce a novel real-world high-resolution dataset with events and color videos named HighREV, which provides a challenging evaluation setting for the examined task. Extensive experiments show that our network consistently outperforms previous state-of-the-art methods on frame interpolation, single image deblurring, and the joint task of both. Experiments on domain transfer reveal that self-supervised training effectively mitigates the performance degradation observed when transitioning from synthetic data to real-world data. Code and datasets are available at https://***/AHupuJR/REFID. Copyright © 2023, The Authors. All rights reserved.

关键词： Self-supervised learning

Object Finding in Cluttered Scenes Using Interactive Perception

学校读者我要写书评

暂无评论

Object Finding in Cluttered Scenes Using Interactive Percept...

IEEE International Conference on Robotics and Automation (ICRA)

作者： Tonci Novkovic Remi Pautrat Fadri Furrer Michel Breyer Roland Siegwart Juan Nieto Autonomous Systems Lab ETH Zurich Switzerland Computer Vision and Geometry Group ETH Zurich Switzerland

ISBN: (数字)9781728173955

ISBN: (纸本)9781728173962

Object finding in clutter is a skill that requires perception of the environment and in many cases physical interaction. In robotics, interactive perception defines a set of algorithms that leverage actions to improve the perception of the environment, and vice versa use perception to guide the next action. Scene interactions are difficult to model, therefore, most of the current systems use predefined heuristics. This limits their ability to efficiently search for the target object in a complex environment. In order to remove heuristics and the need for explicit models of the interactions, in this work we propose a reinforcement learning based active and interactive perception system for scene exploration and object search. We evaluate our work both in simulated and in real-world experiments using a robotic manipulator equipped with an RGB and a depth camera, and compare our system to two baselines. The results indicate that our approach, trained in simulation only, transfers smoothly to reality and can solve the object finding task efficiently and with more than 88% success rate.

关键词： Cameras Task analysis Robot sensing systems Search problems Clutter Detectors