检索结果-内蒙古大学图书馆

CAFE-TALK: GENERATING 3D TALKING FACE ANIMATION WITH MULTIMODAL COARSE- AND FINE-GRAINED CONTROL

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Chen, Hejia Zhang, Haoxian Zhang, Shoulong Liu, Xiaoqiang Zhuang, Sisi Zhang, Yuan Wan, Pengfei Zhang, Di Li, Shuai State Key Laboratory of Virtual Reality Systems and Technology Beihang University China Kuaishou Technology China Zhongguancun Laboratory China

Speech-driven 3D talking face method should offer both accurate lip synchronization and controllable expressions. Previous methods solely adopt discrete emotion labels to globally control expressions throughout sequences while limiting flexible fine-grained facial control within the spatiotemporal domain. We propose a diffusion-transformer-based 3D talking face generation model, Cafe-Talk, which simultaneously incorporates coarse- and fine-grained multimodal control conditions. Nevertheless, the entanglement of multiple conditions challenges achieving satisfying performance. To disentangle speech audio and fine-grained conditions, we employ a two-stage training pipeline. Specifically, Cafe-Talk is initially trained using only speech audio and coarse-grained conditions. Then, a proposed fine-grained control adapter gradually adds fine-grained instructions represented by action units (AUs), preventing unfavorable speech-lip synchronization. To disentangle coarse- and fine-grained conditions, we design a swap-label training mechanism, which enables the dominance of the fine-grained conditions. We also devise a mask-based CFG technique to regulate the occurrence and intensity of fine-grained control. In addition, a text-based detector is introduced with text-AU alignment to enable natural language user input and further support multimodal control. Extensive experimental results prove that Cafe-Talk achieves state-of-the-art lip synchronization and expressiveness performance and receives wide acceptance in fine-grained control in user studies. Project page: https://***/cafe-talk/ © 2025, CC BY.

关键词： Coarse-grained modeling

Refining Boundary Sample Space for Differentiable Rendering 16

学校读者我要写书评

暂无评论

Refining Boundary Sample Space for Differentiable Rendering

2024 16th International Conference on Graphics and Image Processing, ICGIP 2024

作者： Gao, Ruicheng Qi, Yue State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering Beihang University Beijing100191 China Jingdezhen Research Institute Beihang University Jingdezhen333000 China Qingdao Research Institute Beihang University Qingdao266104 China

ISBN: (数字)9781510688780

ISBN: (纸本)9781510688773

The objective of physics-based differentiable rendering (PBDR) is to propagate gradients between scene parameters and the intensities of image pixels in a manner that is physically correct. The gradients obtained can be applied in optimization algorithms for the reconstruction of 3D geometry or materials, or they can be further propagated into neural network to learn neural representations of the scene. However, applying automatic differentiation techniques directly to the primary rendering process will result in biased gradients, as the rendering integral contains moving high-dimensional discontinuities. Based on how these discontinuities are managed-either implicitly or explicitly-existing PBDR methods can be categorized into two groups: reparameterization methods and boundary sampling methods. Boundary sampling methods need to construct paths that have one segment tangent to the geometry being differentiated in order to estimate a boundary integral to address the discontinuities explicitly. Such paths are usually constructed by sampling the tangent segment first and then extending it to complete the paths for subsequent processing. Fortunately, the number of dimensions in the space composed of such tangent segments is only three. In scenes comprised solely of triangle meshes, the first dimension is used to parameterize all the edges on the mesh, which determines a point on the tangent segment. The remaining two dimensions are used to parameterize the direction of the tangent segment. However, state-of-the-art boundary sampling methods parameterize the first dimension uniformly, which is inefficient because only a small portion of the edges contributes to the boundary integral, resulting in wasted parameter space. In this paper, we parameterize the first dimension by considering both edge length and contributions, thereby allocating more parameter space to important edges. Experiments demonstrate that our methods achieve lower variance gradients in the forward dif

关键词： Mesh generation

Research on ABAQUS-Based Fairing Mechanical Specimen Modeling and Mechanical Properties 7th

学校读者我要写书评

暂无评论

Research on ABAQUS-Based Fairing Mechanical Specimen Modelin...

7th China Aeronautical Science and technology Conference, CASTC 2024

作者： Dong, Yushuang Yin, Rongying Xin, Yuhan Yang, Shangru Gao, Hanjun School of Technology Beijing Forestry University Beijing100083 China AVIC HARBN AIRCRAFT INDUSTRY GROUP CO. LTD Harbin150060 China State Key Laboratory of Virtual Reality Technology and Systems School of Mechanical Engineering and Automation Beihang University Beijing100191 China

ISBN: (纸本)9789819797646

To deeply study the mechanical properties of the fairing, the mechanically cut specimens of the fairing were modeled and simulated using ABAQUS 6.14 finite element software. By establishing the geometry of the fairing mechanical specimens, the stress–strain response and deformation characteristics of six mechanical specimens under different loading conditions were simulated and analyzed. The results show the differences in the mechanical properties of the fairing specimens with two different materials, Notably, when aramid paper honeycomb material was juxtaposed with high-temperature fireproof cloth material, the former exhibited superior load-bearing capacity and robustness. In addition, the mechanical properties of the fairing were further investigated in depth by comparing the mechanical responses of the specimens under different gradient forces. © Chinese Society of Aeronautics and Astronautics 2025.

关键词： ABAQUS

Passive activity enhances residual control ability in patients with complete spinal cord injury

学校读者我要写书评

暂无评论

Neural Regeneration Research 2025年第8期20卷 2337-2347页

作者： Yanqing Xiao Mingming Gao Zejia He Jia Zheng Hongming Bai Jia-Sheng Rao Guiyun Song Wei Song Xiaoguang Li Beijing Key Laboratory for Biomaterials and Neural Regeneration Beijing Advanced Innovation Center for Biomedical EngineeringSchool of Biological Science and Medical EngineeringBeihang UniversityBeijingChina Department of Rehabilitation Evaluation China Rehabilitation Research CenterBeijingChina Cardiac Center Beijing Children’s HospitalCapital Medical UniversityBeijingChina The State Key Laboratory of Virtual Reality Technology and Systems Beihang UniversityBeijingChina Department of Rehabilitation Engineering China Rehabilitation Science InstituteBeijingChina

Patients with complete spinal cord injury retain the potential for volitional muscle activity in muscles located below the spinal injury ***,because of prolonged inactivity,initial attempts to activate these muscles may not effectively engage any of the remaining neurons in the descending pathway.A previous study unexpectedly found that a brief clinical round of passive activity significantly increased volitional muscle activation,as measured by surface *** this study,we further explored the effect of passive activity on surface electromyographic signals during volitional control tasks among individuals with complete spinal cord *** patients with chronic complete thoracic spinal cord injury were *** electromyography data from eight major leg muscles were acquired and compared before and after the passive activity *** results indicated that the passive activity led to an increased number of activated volitional muscles and an increased frequency of *** the cumulative root mean square of surface electromyography amplitude for volitional control of movement showed a slight increase after passive activity,the difference was not statistically *** findings suggest that brief passive activity may enhance the ability to initiate volitional muscle activity during surface electromyography tasks and underscore the potential of passive activity for improving residual motor control among patients with motor complete spinal cord injury.

关键词： complete spinal cord injury cycle training epidural electrical stimulation motor training passive activity physiological state spinal cord circuit surface electromyography volitional control task

SSCCPC-Net: Simultaneously Learning 2D and 3D Features with CLIP for Semantic Scene Completion on Point Cloud 41st

学校读者我要写书评

暂无评论

SSCCPC-Net: Simultaneously Learning 2D and 3D Features with...

41st Computer Graphics International Conference, CGI 2024

作者： Duan, Wantong Bao, Yongtang Qi, Yue State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering Beihang University Beijing100191 China Shandong University of Science and Technology Qingdao266590 China Jingdezhen Research Institute of Beihang University Jingdezhen333000 China Qingdao Research Institute of Beihang University Qingdao266000 China

ISBN: (纸本)9783031820236

Compare to traditional Scene Completion (SC), Semantic Scene Completion (SSC) is a challenging task that aims to generate complete and semantically consistent 3D scene from partial and sparse input data, which is fundamental to fully understanding the scene and being able to interact with it. Consequently, the SSC task has received much attention in recent years. Most of the methods are voxel-based approaches, but they have high computational and memory requirements. A few works based on point cloud do not sufficiently exploit the correlation between semantic segmentation and geometric completion subtasks, while focusing too much on point cloud shape features and ignoring the rich texture information that RGB images can provide. In this paper, we present SSCCPC-Net (Semantic Scene Completion with CLIP on Point Cloud-Net), a novel network architecture for point cloud semantic scene completion using a combination of 2D and 3D features. Inspired by recent works of large pretrained vision-language models in semantic segmentation, we explore to accomplish SSC task with the help of Contrastive Language-Image Pre-Training (CLIP) model. Specifically, we use the CLIP features for guidance to fuse the 2D features extracted from the RGB image and the 3D features extracted from the point cloud. The fused features are then fed into our designed Semantic-Completion Decoder for per-point semantic prediction and semantic labeling-assisted point cloud completion. Finally, we obtain the complete semantically point cloud. Numerous experiments have demonstrated that our method has higher effectiveness and generalizability compared to state-of-the-art methods. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

关键词： Semantic Segmentation

Transforming Vision Transformer: Towards Efficient Multi-Task Asynchronous Learning

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Zhong, Hanwen Chen, Jiaxin Zhang, Yutong Huang, Di Wang, Yunhong State Key Laboratory of Virtual Reality Technology and Systems Beihang University China School of Computer Science and Engineering Beihang University Beijing China

Multi-Task Learning (MTL) for Vision Transformer aims at enhancing the model capability by tackling multiple tasks simultaneously. Most recent works have predominantly focused on designing Mixture-of-Experts (MoE) structures and integrating Low-Rank Adaptation (LoRA) to efficiently perform multi-task learning. However, their rigid combination hampers both the optimization of MoE and the effectiveness of reparameterization of LoRA, leading to sub-optimal performance and low inference speed. In this work, we propose a novel approach dubbed Efficient Multi-Task Learning (EMTAL) by transforming a pre-trained Vision Transformer into an efficient multi-task learner during training, and reparameterizing the learned structure for efficient inference. Specifically, we firstly develop the MoEfied LoRA structure, which decomposes the pre-trained Transformer into a low-rank MoE structure and employ LoRA to fine-tune the parameters. Subsequently, we take into account the intrinsic asynchronous nature of multi-task learning and devise a learning Quality Retaining (QR) optimization mechanism, by leveraging the historical high-quality class logits to prevent a well-trained task from performance degradation. Finally, we design a router fading strategy to integrate the learned parameters into the original Transformer, archiving efficient inference. Extensive experiments on public benchmarks demonstrate the superiority of our method, compared to the state-of-the-art multi-task learning approaches. The project page is available at https://***/Yewen1486/EMTAL. Copyright © 2025, The Authors. All rights reserved.

关键词： Multi-task learning

A Survey on Remote Sensing Foundation Models: From Vision to Multimodality

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Huang, Ziyue Yan, Hongxi Zhan, Qiqi Yang, Shuai Zhang, Mingming Zhang, Chenkai Lei, YiMing Liu, Zeming Liu, Qingjie Wang, Yunhong State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing100191 China Hangzhou Innovation Institute Beihang University Hangzhou310051 China

The rapid advancement of remote sensing foundation models, particularly vision and multimodal models, has significantly enhanced the capabilities of intelligent geospatial data interpretation. These models combine various data modalities, such as optical, radar, and LiDAR imagery, with textual and geographic information, enabling more comprehensive analysis and understanding of remote sensing data. The integration of multiple modalities allows for improved performance in tasks like object detection, land cover classification, and change detection, which are often challenged by the complex and heterogeneous nature of remote sensing data. However, despite these advancements, several challenges remain. The diversity in data types, the need for large-scale annotated datasets, and the complexity of multimodal fusion techniques pose significant obstacles to the effective deployment of these models. Moreover, the computational demands of training and fine-tuning multimodal models require significant resources, further complicating their practical application in remote sensing image interpretation tasks. This paper provides a comprehensive review of the state-of-the-art in vision and multimodal foundation models for remote sensing, focusing on their architecture, training methods, datasets and application scenarios. We discuss the key challenges these models face, such as data alignment, cross-modal transfer learning, and scalability, while also identifying emerging research directions aimed at overcoming these limitations. Our goal is to provide a clear understanding of the current landscape of remote sensing foundation models and inspire future research that can push the boundaries of what these models can achieve in real-world applications. The list of resources collected by the paper can be found in the https://***/IRIP-BUAA/A-Review-forremote-sensing-vision-language-models. © 2025, CC BY.

关键词： Photointerpretation

Application of an Improved Residual Attention Neural Network in Mechanical Part Classification 9th

学校读者我要写书评

暂无评论

Application of an Improved Residual Attention Neural Network...

9th International Symposium on Artificial Intelligence and Robotics, ISAIR 2024

作者： Zhang, Tianrui Wang, Aizeng School of Mechanical Engineering and Automation Beihang University Beijing100191 China State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing100191 China

ISBN: (纸本)9789819629138

This paper introduces an improved neural network that integrates residual network and attention mechanisms, achieving a significant improvement of 99.50% accuracy in mechanical part classification tasks. The proposed network enhance accuracy, precision, recall, and F1-score compared to traditional CNNs and other advanced networks in handling complex backgrounds and multi-angle part images. This advancement shows potential in smart manufacturing, offering greater automation, minimizing human intervention. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

关键词： Smart manufacturing

Heterogeneous Mixture of Experts for Remote Sensing Image Super-Resolution

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Chen, Bowen Chen, keyan Yang, Mohan Zou, Zhengxia Shi, Zhenwei Department of Aerospace Intelligent Science and Technology School of Astronautics Beihang University Beijing100191 China State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing100191 China

Remote sensing image super-resolution (SR) aims to reconstruct high-resolution remote sensing images from low-resolution inputs, thereby addressing limitations imposed by sensors and imaging conditions. However, the inherent characteristics of remote sensing images, including diverse ground object types and complex details, pose significant challenges to achieving high-quality reconstruction. Existing methods typically employ a uniform structure to process various types of ground objects without distinction, making it difficult to adapt to the complex characteristics of remote sensing images. To address this issue, we introduce a Mixture of Experts (MoE) model and design a set of heterogeneous experts. These experts are organized into multiple expert groups, where experts within each group are homogeneous while being heterogeneous across groups. This design ensures that specialized activation parameters can be employed to handle the diverse and intricate details of ground objects effectively. To better accommodate the heterogeneous experts, we propose a multi-level feature aggregation strategy to guide the routing process. Additionally, we develop a dual-routing mechanism to adaptively select the optimal expert for each pixel. Experiments conducted on the UCMerced and AID datasets demonstrate that our proposed method achieves superior SR reconstruction accuracy compared to state-of-the-art methods. The code will be available at https://***/Mr-Bamboo/MFG-HMoE. Copyright © 2025, The Authors. All rights reserved.

关键词： Optical resolving power