检索结果-内蒙古大学图书馆

Active Visual Localization for Multi-Agent Collaboration: A Data-Driven Approach

学校读者我要写书评

暂无评论

Active Visual Localization for Multi-Agent Collaboration: A ...

IEEE International Conference on Robotics and Automation (ICRA)

作者： Matthew Hanlon Boyang Sun Marc Pollefeys Hermann Blum Computer Vision and Geometry Lab at ETH Zürich

ISBN: (数字)9798350384574

ISBN: (纸本)9798350384581

Rather than having each newly deployed robot create its own map of its surroundings, the growing availability of SLAM-enabled devices provides the option of simply localizing in a map of another robot or device. In cases such as multi-robot or human-robot collaboration, localizing all agents in the same map is even necessary. However, localizing e.g. a ground robot in the map of a drone or head-mounted MR headset presents unique challenges due to viewpoint changes. This work investigates how active visual localization can be used to overcome such challenges of viewpoint changes. Specifically, we focus on the problem of selecting the optimal viewpoint at a given location. We compare existing approaches in the literature with additional proposed baselines and propose a novel data-driven approach. The result demonstrates the superior performance of our data-driven approach when compared to existing methods, both in controlled simulation experiments and real-world deployment.

关键词： Location awareness Performance evaluation Headphones Visualization Collaboration Robot localization Task analysis

Active Visual Localization for Multi-Agent Collaboration: A Data-Driven Approach

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Hanlon, Matthew Sun, Boyang Pollefeys, Marc Blum, Hermann Computer Vision and Geometry Lab ETH Zürich Switzerland

关键词： Multipurpose robots

A 3D Mixed Reality Interface for Human-Robot Teaming

学校读者我要写书评

暂无评论

A 3D Mixed Reality Interface for Human-Robot Teaming

IEEE International Conference on Robotics and Automation (ICRA)

作者： Jiaqi Chen Boyang Sun Marc Pollefeys Hermann Blum Computer Vision and Geometry Lab ETH Zürich Microsoft Mixed Reality & AI Lab Zürich

ISBN: (数字)9798350384574

ISBN: (纸本)9798350384581

This paper presents a mixed-reality human-robot teaming system. It allows human operators to see in real-time where robots are located, even if they are not in line of sight. The operator can also visualize the map that the robots create of their environment and can easily send robots to new goal positions. The system mainly consists of a mapping and a control module. The mapping module is a real-time multi-agent visual SLAM system that co-localizes all robots and mixed-reality devices to a common reference frame. Visualizations in the mixed-reality device then allow operators to see a virtual life-sized representation of the cumulative 3D map overlaid onto the real environment. As such, the operator can effectively "see through" walls into other rooms. To control robots and send them to new locations, we propose a drag-and-drop interface. An operator can grab any robot hologram in a 3D mini map and drag it to a new desired goal pose. We validate the proposed system through a user study and real-world deployments. We make the mixed-reality application publicly available at ***/cvg/HoloLens_ros.

关键词： Visualization Three-dimensional displays Simultaneous localization and mapping Mixed reality Virtual reality Real-time systems Robots

Semantically Guided Feature Matching for Visual SLAM

学校读者我要写书评

暂无评论

Semantically Guided Feature Matching for Visual SLAM

IEEE International Conference on Robotics and Automation (ICRA)

作者： Oguzhan Ilter Iro Armeni Marc Pollefeys Daniel Barath Computer Vision and Geometry Group ETH Zurich Zurich Switzerland Microsoft Mixed Reality and AI Zurich Lab

ISBN: (数字)9798350384574

ISBN: (纸本)9798350384581

We introduce a new algorithm that utilizes semantic information to enhance feature matching in visual SLAM pipelines. The proposed method constructs a high-dimensional semantic descriptor for each detected ORB feature. When integrated with traditional visual ones, these descriptors aid in establishing accurate tentative point correspondences between consecutive frames. Additionally, our semantic descriptors enrich 3D map points, enhancing loop closure detection by providing deeper insights into the underlying map regions. Experiments on public large-scale datasets demonstrate that our technique surpasses the accuracy of established methods. Importantly, given its detector-agnostic nature, our algorithm also amplifies the efficacy of modern keypoint detectors, such as SuperPoint. The implementation of our algorithm can be found on Github 3 .

关键词： Visualization Accuracy Simultaneous localization and mapping Three-dimensional displays Semantics Pipelines Feature extraction

A 3D Mixed Reality Interface for Human-Robot Teaming

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Chen, Jiaqi Sun, Boyang Pollefeys, Marc Blum, Hermann Computer Vision and Geometry Lab ETH Zürich Switzerland Microsoft Mixed Reality AI Lab Zürich Switzerland

关键词： Mixed reality

Global Localization: Utilizing Relative Spatio-Temporal Geometric Constraints from Adjacent and Distant Cameras

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Altillawi, Mohammad Pataki, Zador Li, Shile Liu, Ziyuan AI Robotics & Simulation Huawei Munich Research Center Germany Computer Vision Center Universitat Autònoma de Barcelona Spain Computer Vision and Geometry Lab ETH Zurich Switzerland

Re-localizing a camera from a single image in a previously mapped area is vital for many computer vision applications in robotics and augmented/virtual reality. In this work, we address the problem of estimating the 6 DoF camera pose relative to a global frame from a single image. We propose to leverage a novel network of relative spatial and temporal geometric constraints to guide the training of a Deep Network for localization. We employ simultaneously spatial and temporal relative pose constraints that are obtained not only from adjacent camera frames but also from camera frames that are distant in the spatio-temporal space of the scene. We show that our method, through these constraints, is capable of learning to localize when little or very sparse ground-truth 3D coordinates are available. In our experiments, this is less than 1% of available ground-truth data. We evaluate our method on 3 common visual localization datasets and show that it outperforms other direct pose estimation methods. © 2023, CC BY-NC-SA.

关键词： Cameras

Long-Term Invariant Local Features via Implicit Cross-Domain Correspondences

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Pataki, Zador Altillawi, Mohammad Kanakis, Menelaos Pautrat, Rémi Shen, Fengyi Liu, Ziyuan Van Gool, Luc Pollefeys, Marc The Computer Vision and Geometry Lab Department of Computer Science ETH Zurich Switzerland The Computer Vision Center CVC-Barcelona The Intelligent Robotics Cloud Technology lab of Huawei-Munich Germany The Computer Vision Lab Department Electrical Engineering ETH Zurich Switzerland The Intelligent Robotics Cloud Technology lab of Huawei-Munich Germany The Intelligent Robotics Cloud Technology lab of Huawei-Munich Germany The Center for Processing Speech and Images KU Leuven The Computer Vision Lab ETH Zurich Switzerland

Modern learning-based visual feature extraction networks perform well in intra-domain localization, however, their performance significantly declines when image pairs are captured across long-term visual domain variations, such as different seasonal and daytime variations. In this paper, our first contribution is a benchmark to investigate the performance impact of long-term variations on visual localization. We conduct a thorough analysis of the performance of current state-of-the-art feature extraction networks under various domain changes and find a significant performance gap between intra- and cross-domain localization. We investigate different methods to close this gap by improving the supervision of modern feature extractor networks. We propose a novel data-centric method, Implicit Cross-Domain Correspondences (iCDC). iCDC represents the same environment with multiple Neural Radiance Fields, each fitting the scene under individual visual domains. It utilizes the underlying 3D representations to generate accurate correspondences across different long-term visual conditions. Our proposed method enhances cross-domain localization performance, significantly reducing the performance gap. When evaluated on popular long-term localization benchmarks, our trained networks consistently outperform existing methods. This work serves as a substantial stride toward more robust visual localization pipelines for long-term deployments, and opens up research avenues in the development of long-term invariant descriptors. Copyright © 2023, The Authors. All rights reserved.

关键词： Feature extraction

Learning-based Relational Object Matching Across Views

学校读者我要写书评

暂无评论

Learning-based Relational Object Matching Across Views

IEEE International Conference on Robotics and Automation (ICRA)

作者： Cathrin Elich Iro Armeni Martin R. Oswald Marc Pollefeys Joerg Stueckler Embodied Vision Group Max Planck Institute for Intelligent Systems Tuebingen Germany Max Planck ETH Center for Learning Systems Computer Vision and Geometry Lab ETH Zurich Switzerland University of Amsterdam Netherlands Microsoft Mixed Reality and AI Lab Zurich Switzerland

Intelligent robots require object-level scene understanding to reason about possible tasks and interactions with the environment. Moreover, many perception tasks such as scene reconstruction, image retrieval, or place recognition can benefit from reasoning on the level of objects. While keypoint-based matching can yield strong results for finding correspondences for images with small to medium view point changes, for large view point changes, matching semantically on the object-level becomes advantageous. In this paper, we propose a learning-based approach which combines local keypoints with novel object-level features for matching object detections between RGB images. We train our object-level matching features based on appearance and inter-frame and cross-frame spatial relations between objects in an associative graph neural network. We demonstrate our approach in a large variety of views on realistically rendered synthetic images. Our approach compares favorably to previous state-of-the-art object-level matching approaches and achieves improved performance over a pure keypoint-based approach for large view-point changes.

关键词：

Learning-based Relational Object Matching Across Views

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Elich, Cathrin Armeni, Iro Oswald, Martin R. Pollefeys, Marc Stueckler, Joerg Embodied Vision Group Max Planck Institute for Intelligent Systems Tuebingen Germany The Max Planck ETH Center for Learning Systems The Computer Vision and Geometry Lab ETH Zurich Switzerland University of Amsterdam Netherlands Microsoft Mixed Reality and AI Lab Zurich Switzerland

关键词： Intelligent robots