检索结果-内蒙古大学图书馆

FSRT: Facial Scene Representation Transformer for Face Reenactment from Factorized Appearance, Head-pose, and Facial Expression Features

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Rochow, Andre Schwarz, Max Behnke, Sven Autonomous Intelligent Systems - Computer Science Institute VI and Center for Robotics University of Bonn Germany Lamarr Institute for Machine Learning and Artificial Intelligence Germany

The task of face reenactment is to transfer the head motion and facial expressions from a driving video to the appearance of a source image, which may be of a different person (cross-reenactment). Most existing methods are CNN-based and estimate optical flow from the source image to the current driving frame, which is then inpainted and refined to produce the output animation. We propose a transformer-based encoder for computing a set-latent representation of the source image(s). We then predict the output color of a query pixel using a transformer-based decoder, which is conditioned with keypoints and a facial expression vector extracted from the driving frame. Latent representations of the source person are learned in a self-supervised manner that factorize their appearance, head pose, and facial expressions. Thus, they are perfectly suited for cross-reenactment. In contrast to most related work, our method naturally extends to multiple source images and can thus adapt to person-specific facial dynamics. We also propose data augmentation and regularization schemes that are necessary to prevent overfitting and support generalizability of the learned representations. We evaluated our approach in a randomized user study. The results indicate superior performance compared to the state-of-the-art in terms of motion transfer quality and temporal consistency.1 Copyright © 2024, The Authors. All rights reserved.

关键词：

LiDAR-Based Registration Against Georeferenced Models for Globally Consistent Allocentric Maps

学校读者我要写书评

暂无评论

LiDAR-Based Registration Against Georeferenced Models for Gl...

IEEE International Workshop on Safety, Security, and Rescue robotics (SSRR)

作者： Jan Quenzel Linus T. Mallwitz Benedikt T. Arnold Sven Behnke Autonomous Intelligent Systems Group Computer Science Institute VI - Intelligent Systems and Robotics Center for Robotics and Lamarr Institute for Machine Learning and Artificial Intelligence University of Bonn Germany Fraunhofer FIT Germany Fraunhofer IAIS Germany

ISBN: (数字)9798331510954

ISBN: (纸本)9798331510961

Modern unmanned aerial vehicles (UAVs) are irreplaceable in search and rescue (SAR) missions to obtain a situational overview or provide closeups without endangering personnel. However, UAVs heavily rely on global navigation satellite system (GNSS) for localization which works well in open spaces, but the precision drastically degrades in the vicinity of buildings. These inaccuracies hinder aggregation of diverse data from multiple sources in a unified georeferenced frame for SAR operators. In contrast, CityGML models provide approximate building shapes with accurate georeferenced poses. Besides, LiDAR works best in the vicinity of 3D structures. Hence, we refine coarse GNSS measurements by registering LiDAR maps against CityGML and digital elevation map (DEM) models as a prior for allocentric mapping. An intuitive plausibility score selects the best hypothesis based on occupancy using a 2D height map. Afterwards, we integrate the registration results in a continuous-time spline-based pose graph optimizer with LiDAR odometry and further sensing modalities to obtain globally consistent, georeferenced trajectories and maps. We evaluate the viability of our approach on multiple flights captured at two distinct testing sites. Our method successfully reduced GNSS offset errors from up-to 16 m to below 0.5 m on multiple flights. Furthermore, we obtain globally consistent maps w.r.t. prior 3D geospatial models.

关键词： Global navigation satellite system Solid modeling Laser radar Three-dimensional displays Buildings Data models Trajectory Geospatial analysis Splines (mathematics) Testing

LiDAR-based Registration against Georeferenced Models for Globally Consistent Allocentric Maps

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Quenzel, Jan Mallwitz, Linus T. Arnold, Benedikt T. Behnke, Sven Autonomous Intelligent Systems Group Computer Science Institute VI – Intelligent Systems and Robotics Germany Center for Robotics Lamarr Institute for Machine Learning and Artificial Intelligence University of Bonn Germany Fraunhofer FIT Germany Fraunhofer IAIS Germany

关键词： Continuous time systems

Learning Embeddings with Centroid Triplet Loss for Object Identification in Robotic Grasping

学校读者我要写书评

暂无评论

Learning Embeddings with Centroid Triplet Loss for Object Id...

IEEE International Conference on Automation science and Engineering (CASE)

作者： Anas Gouda Max Schwarz Christopher Reining Sven Behnke Alice Kirchheim TU Dortmund Lamarr Institute for Machine Learning and Artificial Intelligence Autonomous Intelligent Systems - Computer Science VI & Center for Robotics University of Bonn Germany Fraunhofer IML

ISBN: (数字)9798350358513

ISBN: (纸本)9798350358520

Foundation models are a strong trend in deep learning and computer vision. These models serve as a base for applications as they require minor or no further fine-tuning by developers to integrate into their applications. Foundation models for zero-shot object segmentation such as Segment Anything (SAM) output segmentation masks from images without any further object information. When they are followed in a pipeline by an object identification model, they can perform object detection without training. Here, we focus on training such an object identification model. A crucial practical aspect for an object identification model is to be flexible in input size (number of input images). As object identification is an image retrieval problem, a suitable method should handle multi-query multigallery situations without constraining the number of input images (e.g. by having fixed-size aggregation layers). The key solution to train such a model is the centroid triplet loss (CTL), which aggregates image features to their centroids. CTL yields high accuracy, avoids misleading training signals and keeps the model input size flexible. In our experiments, we establish a new state of the art on the ArmBench object identification task, which shows general applicability of our model. We furthermore demonstrate an integrated unseen object detection pipeline on the challenging HOPE dataset, which requires finegrained detection. There, our pipeline matches and surpasses related methods which have been trained on dataset-specific data. Code and pretrained models are available. 5

关键词： Training Location awareness Image segmentation Accuracy Pipelines Object detection Object segmentation Market research Object recognition Proposals

Learning Embeddings with Centroid Triplet Loss for Object Identification in Robotic Grasping

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Gouda, Anas Schwarz, Max Reining, Christopher Behnke, Sven Kirchheim, Alice TU Dortmund Germany Autonomous Intelligent Systems - Computer Science VI Center for Robotics University of Bonn Germany Lamarr Institute for Machine Learning and Artificial Intelligence Germany Fraunhofer IML Germany

Foundation models are a strong trend in deep learning and computer vision. These models serve as a base for applications as they require minor or no further fine-tuning by developers to integrate into their applications. Foundation models for zero-shot object segmentation such as Segment Anything (SAM) output segmentation masks from images without any further object information. When they are followed in a pipeline by an object identification model, they can perform object detection without training. Here, we focus on training such an object identification model. A crucial practical aspect for an object identification model is to be flexible in input size (number of input images). As object identification is an image retrieval problem, a suitable method should handle multi-query multigallery situations without constraining the number of input images (e.g. by having fixed-size aggregation layers). The key solution to train such a model is the centroid triplet loss (CTL), which aggregates image features to their centroids. CTL yields high accuracy, avoids misleading training signals and keeps the model input size flexible. In our experiments, we establish a new state of the art on the ArmBench object identification task, which shows general applicability of our model. We furthermore demonstrate an integrated unseen object detection pipeline on the challenging HOPE dataset, which requires fine-grained detection. There, our pipeline matches and surpasses related methods which have been trained on dataset-specific data. Code and pretrained models are available. Copyright © 2024, The Authors. All rights reserved.

关键词： Image segmentation

Centroidal State Estimation and Control for Hardware-Constrained Humanoid Robots

学校读者我要写书评

暂无评论

Centroidal State Estimation and Control for Hardware-Constra...

IEEE-RAS International Conference on Humanoid Robots

作者： Grzegorz Ficht Sven Behnke Autonomous Intelligent Systems (AIS) Group Computer Science Institute VI University of Bonn Germany

We introduce novel methods for state estimation, feedforward and feedback control, which specifically target humanoid robots with hardware limitations. Our method com-bines a five-mass model with approximate dynamics of each mass. It enables acquiring an accurate assessment of the centroidal state and center of Pressure, even when direct forms of force or contact sensing are unavailable. Upon this, we develop a feedforward scheme that operates on the centroidal state, accounting for insufficient joint tracking capabilities. Finally, we implement feedback mechanisms, which compensate for the lack in Degrees of Freedom that our NimbRo-OP2X robot has. The whole approach allows for reactive stepping to maintain balance despite these limitations, which was verified on hardware during RoboCup 2023, in Bordeaux, France.

关键词：

Dexterous Pre-grasp Manipulation for Human-like Functional Categorical Grasping: Deep Reinforcement Learning and Grasp Representations

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Pavlichenko, Dmytro Behnke, Sven Group Computer Science Institute VI – Intelligent Systems and Robotics The Center for Robotics The Lamarr Institute for Machine Learning and Artificial Intelligence University of Bonn Germany

Many objects, such as tools and household items, can be used only if grasped in a very specific way—grasped functionally. Often, a direct functional grasp is not possible, though. We propose a method for learning a dexterous pre-grasp manipulation policy to achieve human-like functional grasps using deep reinforcement learning. We introduce a dense multi-component reward function that enables learning a single policy, capable of dexterous pre-grasp manipulation of novel instances of several known object categories with an anthropomorphic hand. The policy is learned purely by means of reinforcement learning from scratch, without any expert demonstrations. It implicitly learns to reposition and reorient objects of complex shapes to achieve given functional grasps. In addition, we explore two different ways to represent a desired grasp: explicit and more abstract, constraint-based. We show that our method consistently learns to successfully manipulate and achieve desired grasps on previously unseen object instances of known categories using both grasp representations. Training is completed on a single GPU in under three hours. Copyright © 2023, The Authors. All rights reserved.

关键词： Deep reinforcement learning

autonomous Fire Fighting with a UAV-UGV Team at MBZIRC 2020

学校读者我要写书评

暂无评论

Autonomous Fire Fighting with a UAV-UGV Team at MBZIRC 2020

2021 International Conference on Unmanned Aircraft systems, ICUAS 2021

作者： Quenzel, Jan Splietker, Malte Pavlichenko, Dmytro Schleich, Daniel Lenz, Christian Schwarz, Max Schreiber, Michael Beul, Marius Behnke, Sven Institute for Computer Science Vi Autonomous Intelligent Systems University of Bonn Friedrich-Hirzebruch-Allee 8 Bonn53115 Germany

ISBN: (纸本)9780738131153

Every day, burning buildings threaten the lives of occupants and first responders trying to save them. Quick action is of essence, but some areas might not be accessible or too dangerous to enter. Robotic systems have become a promising addition to firefighting, but at this stage, they are mostly manually controlled, which is error-prone and requires specially trained personal. We present two systems for autonomous firefighting from air and ground we developed for the Mohamed Bin Zayed International robotics Challenge (MBZIRC) 2020. The systems use LiDAR for reliable localization within narrow, potentially GNSS-restricted environments while maneuvering close to obstacles. Measurements from LiDAR and thermal cameras are fused to track fires, while relative navigation ensures successful extinguishing. We analyze and discuss our successful participation during the MBZIRC 2020, present further experiments, and provide insights into our lessons learned from the competition. © 2021 IEEE.

关键词： Fire extinguishers

Accelerating Interactive Human-like Manipulation Learning with GPU-based Simulation and High-quality Demonstrations

学校读者我要写书评

暂无评论

Accelerating Interactive Human-like Manipulation Learning wi...

IEEE-RAS International Conference on Humanoid Robots

作者： Malte Mosbach Kara Moraw Sven Behnke Autonomous Intelligent Systems (AIS) Group Computer Science Institute VI University of Bonn Germany

ISBN: (纸本)9798350309805

Dexterous manipulation with anthropomorphic robot hands remains a challenging problem in robotics because of the high-dimensional state and action spaces and complex contacts. Nevertheless, skillful closed-loop manipulation is required to enable humanoid robots to operate in unstructured real-world environments. Reinforcement learning (RL) has traditionally imposed enormous interaction data requirements for optimizing such complex control problems. We introduce a new framework that leverages recent advances in GPU-based simulation along with the strength of imitation learning in guiding policy search towards promising behaviors to make RL training feasible in these domains. To this end, we present an immersive virtual reality teleoperation interface designed for interactive human-like manipulation on contact rich tasks and a suite of manipulation environments inspired by tasks of daily living. Finally, we demonstrate the complementary strengths of massively parallel RL and imitation learning, yielding robust and natural behaviors. videos of trained policies, our source code, and the collected demonstration datasets are available at https://***/interactive_human_like_manipulation/.

关键词： Training Solid modeling Source coding Humanoid robots virtual reality Reinforcement learning Aerospace electronics