检索结果-内蒙古大学图书馆

Scene-independent Localization by Learning Residual Coordinate Map with Cascaded Localizers

学校读者我要写书评

暂无评论

Scene-independent Localization by Learning Residual Coordina...

International Symposium on Mixed and Augmented reality (ISMAR)

作者： Junyi Wang Yue Qi State Key Laboratory of Virtual Reality Technology and Systems Beihang University School of Computer Science and Technology Shandong University Qingdao Research Institute of Beihang University

Visual localization plays an essential role in a variety of different fields. The indirect learning based method obtains an excellent performance, but it requests a training process in the target scene before the localization. To achieve deep scene-independent localization, we start by proposing the representation called residual coordinate map between a pair of images. Based on the structure, we put forward a network called SILocNet with the proposed residual coordinate map as the output. The network consists of feature extraction, multi-level feature fusion and transformer based coordinate decoder. Moreover, considering the dynamic scene, we introduce an additional segmentation branch that distinguishes fixed and dynamic parts to promote network perception. With SILocNet in place, a cascaded localizer design is presented for reducing the accumulative error. Meanwhile, the simple mathematical analysis behind the cascaded localizers is also provided. To verify how well our algorithm could perform, we conduct experiments on static 7 Scenes, ScanNet and dynamic TUM RGB-D. In particular, we train the network on ScanNet and test it on 7 Scenes and TUM RGB-D to demonstrate the generality performance. All experiments demonstrate superior performance to other existing methods. Additionally, the effects of the cascaded localizer design, feature fusion, transformer based coordinate decoder and segmentation loss are also discussed.

关键词：

VR-based Vector Watercolor Painting System

学校读者我要写书评

暂无评论

VR-based Vector Watercolor Painting System

virtual reality and 3D User Interfaces Abstracts and Workshops (VRW), IEEE Conference on

作者： Yang Gao Hongming Bai Ziyi Pei Wenfeng Song Aimin Hao State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing China Computer School Beijing Information Science and Technology University Beijing China

We develope a VR-based watercolor painting system. The system generates paintings in the form of vector graphics, which remain clear under free scaling and panoramic projection in VR space. Meanwhile, in order to create a better operation experience and more convenient user interaction in the VR environment, we optimize the system to enable a variety of realistic watercolor effects and provide users with a number of optional painting settings such as color, size, effect, etc. After a short period of study, users can easily use the system to complete the VR painting experience.

关键词： Design engineering Three-dimensional displays Brushes Art Conferences virtual environments Color

Neural 3D Strokes: Creating Stylized 3D Scenes with Vectorized 3D Strokes

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Duan, Hao-Bin Wang, Miao Li, Yan-Xun Yang, Yong-Liang State Key Laboratory of Virtual Reality Technology and Systems SCSE Beihang University China Zhongguanchun Laboratory China Department of Computer Science University of Bath United Kingdom

We present Neural 3D Strokes, a novel technique to generate stylized images of a 3D scene at arbitrary novel views from multi-view 2D images. Different from existing methods which apply stylization to trained neural radiance fields at the voxel level, our approach draws inspiration from image-to-painting methods, simulating the progressive painting process of human artwork with vector strokes. We develop a palette of stylized 3D strokes from basic primitives and splines, and consider the 3D scene stylization task as a multi-view reconstruction process based on these 3D stroke primitives. Instead of directly searching for the parameters of these 3D strokes, which would be too costly, we introduce a differentiable renderer that allows optimizing stroke parameters using gradient descent, and propose a training scheme to alleviate the vanishing gradient issue. The extensive evaluation demonstrates that our approach effectively synthesizes 3D scenes with significant geometric and aesthetic stylization while maintaining a consistent appearance across different views. Our method can be further integrated with style loss and image-text contrastive models to extend its applications, including color transfer and text-driven 3D scene drawing. Results and code are available at http://***/Neural3DStrokes. © 2023, CC BY.

关键词： Gradient methods

Audio-visual aware Foveated Rendering

学校读者我要写书评

暂无评论

IEEE Transactions on Visualization and computer Graphics 2025年 PP卷 PP页

作者： Shi, Xuehuai Li, Yucheng Li, Jiaheng Wu, Jian Yin, Jieming Chen, Xiaobai Wang, Lili Nanjing University of Posts and Telecommunications School of Computer Science Jiangsu 210023 China Beihang University State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering Beijing100191 China Beihang University School of Computer Science and Engineering Beijing100191 China

With the increasing complexity of geometry and rendering effects in virtual reality (VR) scenes, existing foveated rendering methods for VR head-mounted displays (HMDs) struggle to meet users' demands for VR scene rendering with high frame rates (≥ 60 f ps for rendering binocular foveated images in VR scenes containing over 50M triangles). Current research validates that auditory content affects the perception of the human visual system (HVS). However, existing foveated rendering methods primarily model the HVS's eccentricity-dependent visual perception ability on the visual content in VR while ignoring the impact of auditory content on the HVS's visual perception. In this paper, we introduce an auditory-content-based perceived rendering quality analysis to quantify the impact of visual perception under different auditory conditions in foveated rendering. Based on the analysis results, we propose an audio-visual aware foveated rendering method (AvFR). AvFR first constructs an audio-visual feature-driven perception model that predicts the eccentricity-based visual perception in real time by combining the scene's audio-visual content, and then proposes a foveated rendering cost optimization algorithm to adaptively control the shading rate of different regions with the guidance of the perception model. In complex scenes with visual and auditory content containing over 1.17M triangles, AvFR renders high-quality binocular foveated images at an average frame rate of 116 f ps. The results of the main user study and performance evaluation validate that AvFR achieves significant performance improvement (up to 1.4× speedup) without lowering the perceived visual quality compared with the state-of-the-art VR-HMD foveated rendering method. © 1995-2012 IEEE.

关键词： Foveated Rendering Perception-driven Rendering virtual reality

FaceCom: Towards High-fidelity 3D Facial Shape Completion via Optimization and Inpainting Guidance

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Yinglong Wu, Hongyu Wang, Xiaogang Qin, Qingzhao Zhao, Yijiao Wang, Yong Hao, Aimin State Key Laboratory of Virtual Reality Technology and Systems Beihang University China Peking University School Hospital of Stomatology China College of Computer and Information Science Southwest University China

We propose FaceCom, a method for 3D facial shape completion, which delivers high-fidelity results for incomplete facial inputs of arbitrary forms. Unlike end-to-end shape completion methods based on point clouds or voxels, our approach relies on a mesh-based generative network that is easy to optimize, enabling it to handle shape completion for irregular facial scans. We first train a shape generator on a mixed 3D facial dataset containing 2405 identities. Based on the incomplete facial input, we fit complete faces using an optimization approach under image inpainting guidance. The completion results are refined through a post-processing step. FaceCom demonstrates the ability to effectively and naturally complete facial scan data with varying missing regions and degrees of missing areas. Our method can be used in medical prosthetic fabrication and the registration of deficient scanning data. Our experimental results demonstrate that FaceCom achieves exceptional performance in fitting and shape completion tasks. The code is available at https://***/dragonylee/***. Copyright © 2024, The Authors. All rights reserved.

关键词： Air navigation

APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for vision Transformers

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Wu, Zhuguanyu Zhang, Jiayi Chen, Jiaxin Guo, Jinyang Huang, Di Wang, Yunhong State Key Laboratory of Virtual Reality Technology and Systems Beihang University China School of Computer Science and Engineering Beihang University Beijing China School of Artificial Intelligence Beihang University Beijing China

vision Transformers (ViTs) have become one of the most commonly used backbones for vision tasks. Despite their remarkable performance, they often suffer significant accuracy drops when quantized for practical deployment, particularly by post-training quantization (PTQ) under ultra-low bits. Recently, reconstruction-based PTQ methods have shown promising performance in quantizing Convolutional Neural Networks (CNNs). However, they fail when applied to ViTs, primarily due to the inaccurate estimation of output importance and the substantial accuracy degradation in quantizing post-GELU activations. To address these issues, we propose APHQ-ViT, a novel PTQ approach based on importance estimation with Average Perturbation Hessian (APH). Specifically, we first thoroughly analyze the current approximation approaches with Hessian loss, and propose an improved average perturbation Hessian loss. To deal with the quantization of the post-GELU activations, we design an MLP Reconstruction (MR) method by replacing the GELU function in MLP with ReLU and reconstructing it by the APH loss on a small unlabeled calibration set. Extensive experiments demonstrate that APHQ-ViT using linear quantizers outperforms existing PTQ methods by substantial margins in 3-bit and 4-bit across different vision tasks. The source code is available at https://***/GoatWu/APHQ-ViT. Copyright © 2025, The Authors. All rights reserved.

关键词： Convolutional neural networks

Dual Temporal Transformers for Fine-Grained Dangerous Action Recognition

学校读者我要写书评

暂无评论

Dual Temporal Transformers for Fine-Grained Dangerous Action...

IEEE International Conference on Image Processing

作者： Wenfeng Song Xingliang Jin Yang Ding Yang Gao Xia Hou Computer School of Beijing Information Science and Technology University State Key Laboratory of Virtual Reality Technology and Systems Beihang University Research Unit of Virtual Human and Virtual Surgery (2019RU004) Chinese Academy of Medical Sciences

Recognizing dangerous actions is a critical task in computer vision, especially for surveillance applications. While existing deep learning methods have been successful in confined environments, they struggle with the anomalous and salient variations of human postures in dangerous actions. Additionally, finer-grained dangerous actions require more discriminative cues, adding to the complexity of the task. To address these challenges, we propose a novel solution that models the intrinsic and invariant properties of dangerous actions at multiple temporal semantic levels. Concretely, we propose a Dual Temporal Transformers (DTT) to capture temporal interactions between distinct key points in the human body aggregation from shallow to deep layers, increasing the perception field from local to global, simultaneously. By doing so, our method avoids overfitting to unrelated or minor clues in videos and achieves a generalized representation of abnormal actions. We evaluate our approach on indoor and outdoor environments and found that DTT outperforms existing methods in terms of efficiency and accuracy. Our code and dataset are pubic available on https://***/AveryJohnsonJJ/***.

关键词：

PHA: Patch-Wise High-Frequency Augmentation for Transformer-Based Person Re-Identification

学校读者我要写书评

暂无评论

PHA: Patch-Wise High-Frequency Augmentation for Transformer-...

Conference on computer vision and Pattern Recognition (CVPR)

作者： Guiwei Zhang Yongfei Zhang Tianyu Zhang Bo Li Shiliang Pu Beijing Key Laboratory of Digital Media School of Computer Science and Engineering Beihang University State Key Laboratory of Virtual Reality Technology and Systems Beihang University Pengcheng Laboratory Hikvision Research Institute

Although recent studies empirically show that injecting Convolutional Neural Networks (CNNs) into vision Transformers (ViTs) can improve the performance of person reidentification, the rationale behind it remains elusive. From a frequency perspective, we reveal that ViTs perform worse than CNNs in preserving key high-frequency components (e.g, clothes texture details) since high-frequency components are inevitably diluted by low-frequency ones due to the intrinsic Self-Attention within ViTs. To remedy such inadequacy of the ViT, we propose a Patch-wise High-frequency Augmentation (PHA) method with two core designs. First, to enhance the feature representation ability of high-frequency components, we split patches with high-frequency components by the Discrete Haar Wavelet Transform, then empower the ViT to take the split patches as auxiliary input. Second, to prevent high-frequency components from being diluted by low-frequency ones when taking the entire sequence as input during network optimization, we propose a novel patch-wise contrastive loss. From the view of gradient optimization, it acts as an implicit augmentation to improve the representation ability of key high-frequency components. This benefits the ViT to capture key high-frequency components to extract discriminative person representations. PHA is necessary during training and can be removed during inference, without bringing extra complexity. Extensive experiments on widely-used ReID datasets validate the effectiveness of our method.

关键词：

Generative Face Parsing Map Guided 3D Face Reconstruction Under Occluded Scenes

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhao, Dapeng Qi, Yue State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering Beihang University Beijing China Peng Cheng Laboratory Shenzhen China Qingdao Research Institute of Beihang University Qingdao China

Over the past few years, single-view 3D face reconstruction methods can produce beautiful 3D models. Nevertheless, the input of these works is unobstructed faces. We describe a system designed to reconstruct convincing face texture in the case of occlusion. Motivated by parsing facial features, we propose a complete face parsing map generation method guided by landmarks. We estimate the 2D face structure of the reasonable position of the occlusion area, which is used for the construction of 3D texture. An excellent anti-occlusion face reconstruction method should ensure the authenticity of the output, including the topological structure between the eyes, nose, and mouth. We extensively tested our method and its components, qualitatively demonstrating the rationality of our estimated facial structure. We conduct extensive experiments on general 3D face reconstruction tasks as concrete examples to demonstrate the method’s superior regulation ability over existing methods often break down. We further provide numerous quantitative examples showing that our method advances both the quality and the robustness of 3D face reconstruction under occlusion scenes. Copyright © 2024, The Authors. All rights reserved.

关键词： 3D reconstruction