检索结果-内蒙古大学图书馆

IEEE Annual International Symposium Virtual reality

作者： Hannah Schieber Jacob Young Tobias Langlotz Stefanie Zollmann Daniel Roth Technical University of Munich Human-Centered Computing and Extended Reality Lab TUM University Hospital Clinic for Orthopedics and Sports Orthopedics Munich Germany Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) Erlangen Germany Department of Computer Science University of Otago Dunedin New Zealand Department of Computer Science Aarhus University Aarhus Denmark

ISBN: (数字)9798331536459

ISBN: (纸本)9798331536466

Advancements in 3D rendering like Gaussian Splatting (GS) allow novel view synthesis and real-time rendering in virtual reality (VR). However, GS-created 3D environments are often difficult to edit. For scene enhancement or to incorporate 3D assets, segmenting Gaussians by class is essential. Existing segmentation approaches are typically limited to certain types of scenes, e.g., "circular" scenes, to determine clear object boundaries. However, this method is ineffective when removing large objects in non-"circling" scenes such as large outdoor *** propose Semantics-Controlled GS (SCGS), a segmentation-driven GS approach, enabling the separation of large scene parts in uncontrolled, natural environments. SCGS allows scene editing and the extraction of scene parts for VR. Additionally, we introduce a challenging outdoor dataset, overcoming the "circling" setup. We outperform the state-of-the-art in visual quality on our dataset and in segmentation quality on the 3D-OVS dataset. We conducted an exploratory user study, comparing a 360-video, plain GS, and SCGS in VR with a fixed viewpoint. In our subsequent main study, users were allowed to move freely, evaluating plain GS and SCGS. Our main study results show that participants clearly prefer SCGS over plain GS. We overall present an innovative approach that surpasses the state-of-the-art both technically and in user experience.

关键词： Visualization Three-dimensional displays Semantics Virtual reality User interfaces Rendering (computer graphics) User experience Real-time systems

来源：评论

学校读者我要写书评

暂无评论

A Modular Approach for 3D Reconstruction with Point Cloud Overlay

A Modular Approach for 3D Reconstruction with Point Cloud Ov...

引用

IEEE International Symposium on Mixed and Augmented reality Workshops (ISMARW)

作者： Hannah Schieber Fritz Schmid Mubashir-UI-Hassan Stefanie Zollmann Daniel Roth Human-Centered Computing and Extended Reality Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) Department of Computer Science University of Otago Dunedin New Zealand School of Medicine and Health Klinikum Rechts der Isar Orthopaedics and Sports Orthopaedics Technical University of Munich

Integrating the real-world through the power of 3D reconstruction plays an essential role in extended reality teleconsultation. Video streams and voxel-based representations are common techniques for teleconsultation visualization. However, video streaming applications miss 3D content information, and voxel-based representation can only provide visual details with increasing computational costs. To enhance the classical voxel-based representation and address the timeliness of voxels, we present a modular approach which provides a point cloud overlay to highlight the latest update in the current camera frustum. Our approach applies state-of-the-art algorithms and custom packages within ROS to enable 3D reconstruction with dynamic cameras. We present a modular approach allowing the flexible exchange of the individual part, i.e. the camera or SLAM algorithm. This work presents results from a pilot study involving five participants to gain an impression of what kind of visualization type would be preferred and whether the point cloud overlay would assist the user in recognizing changes in the surroundings. The point cloud overlay enabled the participants to perceive more changes. The pilot study revealed that 60% of the participants showed a preference for the point cloud overlay over the pure mesh representation.

关键词：

来源：评论

学校读者我要写书评

暂无评论

GBOT: Graph-Based 3D Object Tracking for Augmented reality-Assisted Assembly Guidance

GBOT: Graph-Based 3D Object Tracking for Augmented Reality-A...

引用

IEEE Annual International Symposium Virtual reality

作者： Shiyu Li Hannah Schieber Niklas Corell Bernhard Egger Julian Kreimeier Daniel Roth Department Clinical Medicine Machine Intelligence in Orthopedics Clinic for Orthopedics and Sports Orthopedics School of Medicine and Health Technical University of Munich Human-Centered Computing and Extended Reality Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) Erlangen Germany Lehrstuhl für Graphische Datenverarbeitung (LGDV) Friedrich-Alexander Universität (FAU) Erlangen-Nürnberg Erlangen Germany

Guidance for assemblable parts is a promising field for augmented reality. Augmented reality assembly guidance requires 6D object poses of target objects in real time. Especially in time-critical medical or industrial settings, continuous and markerless tracking of individual parts is essential to visualize instructions superimposed on or next to the target object parts. In this regard, occlusions by the user’s hand or other objects and the complexity of different assembly states complicate robust and real-time markerless multi-object tracking. To address this problem, we present Graph-based Object Tracking (GBOT), a novel graph-based single-view RGB-D tracking approach. The real-time markerless multi-object tracking is initialized via 6D pose estimation and updates the graph-based assembly poses. The tracking through various assembly states is achieved by our novel multi-state assembly graph. We update the multi-state assembly graph by utilizing the relative poses of the individual assembly parts. Linking the individual objects in this graph enables more robust object tracking during the assembly process. For evaluation, we introduce a synthetic dataset of publicly available and 3D printable assembly assets as a benchmark for future work. Quantitative experiments in synthetic data and further qualitative study in real test data show that GBOT can outperform existing work towards enabling context-aware augmented reality assembly guidance. Dataset and code will be made publically available. ** ** https://***/roth-hex-lab/gbot

关键词：

来源：评论

学校读者我要写书评

暂无评论

DynaMoN: Motion-Aware Fast and Robust Camera Localization for Dynamic Neural Radiance Fields

arXiv

引用

arXiv 2023年

作者： Schischka, Nicolas Schieber, Hannah Karaoglu, Mert Asim Gorgulu, Melih Grötzner, Florian Ladikos, Alexander Roth, Daniel Navab, Nassir Busam, Benjamin Technical University of Munich Munich Germany Human-Centered Computing and Extended Reality Lab Technical University of Munich School of Medicine and Health Klinikum rechts der Isar Orthopedics and Sports Orthopedics Munich Germany Friedrich-Alexander Universität Erlangen-Nürnberg Erlangen Germany ImFusion GmbH Munich Germany Johns Hopkins University BaltimoreMD United States

The accurate reconstruction of dynamic scenes with neural radiance fields is significantly dependent on the estimation of camera poses. Widely used structure-from-motion pipelines encounter difficulties in accurately tracking the camera trajectory when faced with separate dynamics of the scene content and the camera movement. To address this challenge, we propose Dynamic Motion-Aware Fast and Robust Camera Localization for Dynamic Neural Radiance Fields (DynaMoN). DynaMoN utilizes semantic segmentation and generic motion masks to handle dynamic content for initial camera pose estimation and statics-focused ray sampling for fast and accurate novel-view synthesis. Our novel iterative learning scheme switches between training the NeRF and updating the pose parameters for an improved reconstruction and trajectory estimation quality. The proposed pipeline shows significant acceleration of the training process. We extensively evaluate our approach on two real-world dynamic datasets, the TUM RGB-D dataset and the BONN RGB-D Dynamic dataset. DynaMoN improves over the state-of-the-art both in terms of reconstruction quality and trajectory accuracy. We plan to make our code public to enhance research in this area. Code available: https://***/DynaMoN/. © 2023, CC BY.

关键词： Semantic Segmentation

来源：评论

学校读者我要写书评

暂无评论

ASDF: Assembly State Detection Utilizing Late Fusion by Integrating 6D Pose Estimation

ASDF: Assembly State Detection Utilizing Late Fusion by Inte...

引用

International Symposium on Mixed and Augmented reality (ISMAR)

作者： Hannah Schieber Shiyu Li Niklas Corell Philipp Beckerle Julian Kreimeier Daniel Roth Technical University of Munich Human-Centered Computing and Extended Reality Lab TUM School of Medicine and Health TUM School of Computation Information and Technology Clinic for Orthopedics and Sports Orthopedics TUM University Hospital Munich Germany Department Artificial Intelligence in Biomedical Engineering Friedrich-Alexander Universität (FAU) Erlangen-Nürnberg Erlangen Germany Chair of Autonomous Systems and Mechatronics Friedrich-Alexander Universität (FAU) Erlangen-Nürnberg Erlangen Germany

ISBN: (数字)9798331516475

ISBN: (纸本)9798331516482

In medical and industrial domains, providing guidance for assembly processes can be critical to ensure efficiency and safety. Errors in assembly can lead to significant consequences such as extended surgery times and prolonged manufacturing or maintenance times in industry. Assembly scenarios can benefit from in-situ augmented reality visualization, i.e., augmentations in close proximity to the target object, to provide guidance, reduce assembly times, and minimize errors. In order to enable in-situ visualization, 6D pose estimation can be leveraged to identify the correct location for an augmentation. Existing 6D pose estimation techniques primarily focus on individual objects and static captures. However, assembly scenarios have various dynamics, including occlusion during assembly and dynamics in the appearance of assembly objects. Existing work focus either on object detection combined with state detection, or focus purely on the pose estimation. To address the challenges of 6D pose estimation in combination with assembly state detection, our approach ASDF builds upon the strengths of YOLOv8, a real-time capable object detection framework. We extend this framework, refine the object pose, and fuse pose knowledge with network-detected pose information. Utilizing our late fusion in our Pose2State module results in refined 6D pose estimation and assembly state detection. By combining both pose and state information, our Pose2State module predicts the final assembly state with precision. The evaluation of our ASDF dataset shows that our Pose2State module leads to an improved assembly state detection and that the improvement of the assembly state further leads to a more robust 6D pose estimation. Moreover, on the GBOT dataset, we outperform the pure deep learning-based network and even outperform the hybrid and pure tracking-based approaches.

关键词： Knowledge engineering Visualization Pose estimation Surgery Object detection Real-time systems Safety Maintenance Assembly Augmented reality

来源：评论

学校读者我要写书评

暂无评论

Complementary Textures. A Novel Approach to Object Alignment in Mixed reality

arXiv

引用

arXiv 2022年

作者： Martin-Gomez, Alejandro Winkler, Alexander de la Tijera Obert, Rafael Fotouhi, Javad Roth, Daniel Eck, Ulrich Navab, Nassir The Laboratory for Computational Sensing and Robotics Johns Hopkins University United States The Chair for Computer Aided Medical Procedures Technical University of Munich Germany Philips Research North America United States The Human-Centered Computing and Extended Reality Lab Friedrich-Alexander-Universität Erlangen-Nürnberg Germany

Alignment between real and virtual objects is a challenging task required for the deployment of Mixed reality (MR) into manufacturing, medical, and construction applications. To face this challenge, a series of methods have been proposed. While many approaches use dynamic augmentations such as animations, arrows, or text to assist users, they require tracking the position of the real objects. In contrast, when tracking of the real objects is not available or desired, alternative approaches use virtual replicas of real objects to allow for interactive, perceptual virtual-to-real and/or real-to-virtual alignment. In these cases, the accuracy achieved strongly depends on the quality of the perceptual information provided to the user. This paper proposes a novel set of perceptual alignment concepts that go beyond the use of traditional visualization of the virtual replicas, introducing the concept of COMPLEMENTARY TEXTURES to improve interactive alignment in MR applications. To showcase the advantages of using COMPLEMENTARY TEXTURES, we describe three different implementations that provide highly salient visual cues when misalignment is observed;or present semantic augmentations that, when combined with a real object, provide contextual information that can be used during the alignment process. The authors aim at opening new paths for the community to explore rather than describing end-to-end solutions. The objective is to show multitude of opportunities such concepts could provide for further research and development. © 2022, CC BY.

关键词： Augmented reality

来源：评论

学校读者我要写书评

暂无评论

Deep Sensor Fusion with Pyramid Fusion Networks for 3D Semantic Segmentation

Deep Sensor Fusion with Pyramid Fusion Networks for 3D Seman...

引用

IEEE Symposium on Intelligent Vehicle

作者： Hannah Schieber Fabian Duerr Torsten Schoen Jü rgen Beyerer Human-Centered Computing and Extended Reality Friedrich-Alexander University (FAU) Erlangen-N&#x00FC rnberg Erlangen Germany Vision and Fusion Laboratory Karlsruhe Institute of Technology Karlsruhe Germany Research Institute Almotion Bavaria Technische Hochschule Ingolstadt Ingolstadt Germany Fraunhofer Institute of Optronics System Technologies and Image Exploitation (IOSB) Fraunhofer Center of Machine Learning Karlsruhe Germany

Robust environment perception for autonomous vehicles is a tremendous challenge, which makes a diverse sensor set with e.g. camera, lidar and radar crucial. In the process of understanding the recorded sensor data, 3D semantic segmentation plays an important role. Therefore, this work presents a pyramid-based deep fusion architecture for lidar and camera to improve 3D semantic segmentation of traffic scenes. Individual sensor backbones extract feature maps of camera images and lidar point clouds. A novel Pyramid Fusion Backbone fuses these feature maps at different scales and combines the multimodal features in a feature pyramid to compute valuable multimodal, multi-scale features. The Pyramid Fusion Head aggregates these pyramid features and further refines them in a late fusion step, incorporating the final features of the sensor backbones. The approach is evaluated on two challenging outdoor datasets and different fusion strategies and setups are investigated. It outperforms recent range view based lidar approaches as well as all so far proposed fusion strategies and architectures.

关键词： Point cloud compression Image segmentation Laser radar Three-dimensional displays Head Fuses Semantics

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：