检索结果-内蒙古大学图书馆

Efficient Region-Aware Neural Radiance Fields for High-Fidelity Talking Portrait Synthesis

学校读者我要写书评

暂无评论

Efficient Region-Aware Neural Radiance Fields for High-Fidel...

International Conference on computer Vision (ICCV)

作者： Jiahe Li Jiawei Zhang Xiao Bai Jun Zhou Lin Gu School of Computer Science and Engineering State Key Laboratory of Software Development Environment Jiangxi Research Institute Beihang University School of Information and Communication Technology Griffith University RIKEN AIP The University of Tokyo

This paper presents ER-NeRF, a novel conditional Neural Radiance Fields (NeRF) based architecture for talking portrait synthesis that can concurrently achieve fast convergence, real-time rendering, and state-of-the-art performance with small model size. Our idea is to explicitly exploit the unequal contribution of spatial regions to guide talking portrait modeling. Specifically, to improve the accuracy of dynamic head reconstruction, a compact and expressive NeRF-based Tri-Plane Hash Representation is introduced by pruning empty spatial regions with three planar hash encoders. For speech audio, we propose a Region Attention Module to generate region-aware condition feature via an attention mechanism. Different from existing methods that utilize an MLP-based encoder to learn the cross-modal relation implicitly, the attention mechanism builds an explicit connection between audio features and spatial regions to capture the priors of local motions. Moreover, a direct and fast Adaptive Pose Encoding is introduced to optimize the head-torso separation problem by mapping the complex transformation of the head pose into spatial coordinates. Extensive experiments demonstrate that our method renders better high-fidelity and audio-lips synchronized talking portrait videos, with realistic details and high efficiency compared to previous methods. Code is available at https://***/Fictionarry/ER-NeRF.

关键词：

Solving Math Word Problems Following Logically Consistent Template

学校读者我要写书评

暂无评论

Solving Math Word Problems Following Logically Consistent Te...

International Joint Conference on Neural Networks (IJCNN)

作者： Zeyu Huang Xiaofeng Zhang Jun Bai Wenge Rong Yuanxin Ouyang Zhang Xiong State Key Laboratory of Software Development Environment Beihang University Beijing China Sino-French Engineer School Beihang University Beijing China School of Computer Science and Engineering Beihang University Beijing China

Solving math word problems (MWPs) is a challenging task. Some existing solvers retrieve textually similar problems and draw on their solutions to solve the given problem. However, textually similar questions are not guaranteed to have similar solutions, and vice versa. Therefore, this work investigates the logical consistency among different problems and proposes a novel framework to solve math word problems following logically consistent templates. Experimental results show that our method outperforms many strong baselines, including some pre-trained language model-based methods. Further analysis shows that our retrieval method does learn the logical similarity between Questions and plays a crucial role in our model's nerformance.

关键词：

Subgraph-Oriented Heterogeneous Drug-Target Interaction Identification

学校读者我要写书评

暂无评论

Subgraph-Oriented Heterogeneous Drug-Target Interaction Iden...

International Joint Conference on Neural Networks (IJCNN)

作者： Xiaofeng Zhang Zeyu Huang Jun Bai Wenge Rong Yuanxin Ouyang Zhang Xiong State Key Laboratory of Software Development Environment Beihang University Beijing China Sino-French Engineer School Beihang University Beijing China School of Computer Science and Engineering Beihang University Beijing China

Drug-target interaction (DTI) is an important task in drug discovery and drug repurposing. Currently, most methods utilizing drug-based and protein-based similarity values to predict DTIs achieve promising results. However, calculating similarities for each pair of nodes is time-consuming, especially for relatively large datasets. In this research, we propose a novel subgraph-oriented heterogeneous DTI identification method that transforms the DTI task from a link prediction task to a subgraph classification task. For each link, a local subgraph around this link is extracted. Then, a subgraph labeling process distinguishes different topologies of subgraphs. A random walk-based node representation generation is also integrated with the model. Finally, we apply a graph neural network for the subgraph classification. Our method avoids incorporating human-made similarity values by extracting more meaningful local subgraph topological information. Experimental studies for known DTI predictions on two DTI datasets show promising results for DTI prediction. Empirical results for new DTI predictions on two external public databases show the generalization ability of the proposed method.

关键词：

Leveraging Predicate and Triplet Learning for Scene Graph Generation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Jiankai Wang, Yunhong Guo, Xiefan Yang, Ruijie Li, Weixin IRIP Lab School of Computer Science and Engineering Beihang University Beijing China State Key Laboratory of Complex & Critical Software Environment Beihang University Beijing China Shanghai Artificial Intelligence Laboratory Shanghai China

Scene Graph Generation (SGG) aims to identify entities and predict the relationship triplets in visual scenes. Given the prevalence of large visual variations of subject-object pairs even in the same predicate, it can be quite challenging to model and refine predicate representations directly across such pairs, which is however a common strategy adopted by most existing SGG methods. We observe that visual variations within the identical triplet are relatively small and certain relation cues are shared in the same type of triplet, which can potentially facilitate the relation learning in SGG. Moreover, for the long-tail problem widely studied in SGG task, it is also crucial to deal with the limited types and quantity of triplets in tail predicates. Accordingly, in this paper, we propose a Dual-granularity Relation Modeling (DRM) network to leverage fine-grained triplet cues besides the coarse-grained predicate ones. DRM utilizes contexts and semantics of predicate and triplet with Dual-granularity Constraints, generating compact and balanced representations from two perspectives to facilitate relation recognition. Furthermore, a Dual-granularity Knowledge Transfer (DKT) strategy is introduced to transfer variation from head predicates/triplets to tail ones, aiming to enrich the pattern diversity of tail classes to alleviate the long-tail problem. Extensive experiments demonstrate the effectiveness of our method, which establishes new state-of-the-art performance on Visual Genome, Open Image, and GQA datasets. Our code is available at https://***/jkli1998/DRM Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics

Leveraging Predicate and Triplet Learning for Scene Graph Generation

学校读者我要写书评

暂无评论

Leveraging Predicate and Triplet Learning for Scene Graph Ge...

Conference on computer Vision and Pattern Recognition (CVPR)

作者： Jiankai Li Yunhong Wang Xiefan Guo Ruijie Yang Weixin Li IRIP Lab School of Computer Science and Engineering Beihang University Beijing China State Key Laboratory of Complex & Critical Software Environment Beihang University Beijing China Shanghai Artificial Intelligence Laboratory Shanghai China

ISBN: (数字)9798350353006

ISBN: (纸本)9798350353013

关键词： Visualization Head Triples (Data structure) Semantics Genomics Tail Pattern recognition

Efficient Data Valuation Approximation in Federated Learning: A Sampling-based Approach

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Wei, Shuyue Tong, Yongxin Zhou, Zimu He, Tianran Xu, Yi State Key Laboratory of Complex & Critical Software Environment Lab School of Computer Science and Engineering Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing Beihang University China City University of Hong Kong Hong Kong

Federated learning (FL) has emerged as a prominent distributed learning paradigm to utilize datasets across multiple data providers. In FL, cross-silo data providers often hesitate to share their high-quality dataset unless their data value can be fairly assessed. Shapley value (SV) has been advocated as the standard metric for data valuation in FL due to its desirable properties. However, the computational overhead of SV is prohibitive in practice, as it inherently requires training and evaluating an FL model across an exponential number of dataset combinations. Furthermore, existing solutions fail to achieve high accuracy and efficiency, making practical use of SV still out of reach, because they ignore choosing suitable computation scheme for approximation framework and overlook the property of utility function in FL. We first propose a unified stratified-sampling framework for two widely-used schemes. Then, we analyze and choose the more promising scheme under the FL linear regression assumption. After that, we identify a phenomenon termed key combinations, where only limited dataset combinations have a high-impact on final data value. Building on these insights, we propose a practical approximation algorithm, IPSS, which strategically selects high-impact dataset combinations rather than evaluating all possible combinations, thus substantially reducing time cost with minor approximation error. Furthermore, we conduct extensive evaluations on the FL benchmark datasets to demonstrate that our proposed algorithm outperforms a series of representative baselines in terms of efficiency and effectiveness. Copyright © 2025, The Authors. All rights reserved.

关键词： Federated learning

Unsupervised Recognition of Unknown Objects for Open-World Object Detection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Fang, Ruohuan Pang, Guansong Zhou, Lei Bai, Xiao Zheng, Jin School of Computer Science and Engineering State Key Laboratory of Software Development Environment Jiangxi Research Institute Beihang University China School of Computing and Information Systems Singapore Management University Singapore School of Computer Science and Artificial Intelligence Wuhan University of Technology China

Open-World Object Detection (OWOD) extends object detection problem to a realistic and dynamic scenario, where a detection model is required to be capable of detecting both known and unknown objects and incrementally learning newly introduced knowledge. Current OWOD models, such as ORE and OW-DETR, focus on pseudo-labeling regions with high objectness scores as unknowns, whose performance relies heavily on the supervision of known objects. While they can detect the unknowns that exhibit similar features to the known objects, they suffer from a severe label bias problem that they tend to detect all regions (including unknown object regions) that are dissimilar to the known objects as part of the background. To eliminate the label bias, this paper proposes a novel approach that learns an unsupervised discriminative model to recognize true unknown objects from raw pseudo labels generated by unsupervised region proposal methods. The resulting model can be further refined by a classification-free self-training method which iteratively extends pseudo unknown objects to the unlabeled regions. Experimental results show that our method 1) significantly outperforms the prior SOTA in detecting unknown objects while maintaining competitive performance of detecting known object classes on the MS COCO dataset, and 2) achieves better generalization ability on the LVIS and Objects365 datasets. Code is available at https://***/frh23333/mepu-owod. Copyright © 2023, The Authors. All rights reserved.

关键词： Object detection

Adaptive Sparse Convolutional Networks with Global Context Enhancement for Faster Object Detection on Drone Images

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Du, Bowei Huang, Yecheng Chen, Jiaxin Huang, Di State Key Laboratory of Software Development Environment Beihang University Beijing China School of Computer Science and Engineering Beihang University Beijing China Hangzhou Innovation Institute Beihang University Hangzhou China

Object detection on drone images with low-latency is an important but challenging task on the resource-constrained unmanned aerial vehicle (UAV) platform. This paper investigates optimizing the detection head based on the sparse convolution, which proves effective in balancing the accuracy and efficiency. Nevertheless, it suffers from inadequate integration of contextual information of tiny objects as well as clumsy control of the mask ratio in the presence of foreground with varying scales. To address the issues above, we propose a novel global context-enhanced adaptive sparse convolutional network (CEASC). It first develops a context-enhanced group normalization (CE-GN) layer, by replacing the statistics based on sparsely sampled features with the global contextual ones, and then designs an adaptive multilayer masking strategy to generate optimal mask ratios at distinct scales for compact foreground coverage, promoting both the accuracy and efficiency. Extensive experimental results on two major benchmarks, i.e. VisDrone and UAVDT, demonstrate that CEASC remarkably reduces the GFLOPs and accelerates the inference procedure when plugging into the typical state-of-the-art detection frameworks (e.g. RetinaNet and GFL V1) with competitive performance. Code is available at https://***/Cuogeihong/CEASC. Copyright © 2023, The Authors. All rights reserved.

关键词： Object detection

Diving into Continual Ultra-fine-grained Visual Categorization

学校读者我要写书评

暂无评论

Diving into Continual Ultra-fine-grained Visual Categorizati...

Proceedings of the Digital Image Computing: Technqiues and Applications (DICTA)

作者： Pengcheng Zhang Xiaohan Yu Xiao Bai Jin Zheng Xiaoyu Wu Yongsheng Gao School of Computer Science and Engineering State Key Laboratory of Software Development Environment Jiangxi Research Institute Beihang University Beijing China Institute for Integrated and Intelligent Systems Griffith University Nathan Australia

Recent advance in ultra-fine-grained visual categorization (ultra-FGVC) has significantly boosted the capability of deep neural networks for ultra-FGVC tasks. However, building models for continually learning to recognize increasing ultra-fine-grained categories is still under-explored. This limits the application of ultra-FGVC techniques in real-world production. To this, we take the first attempt for continual ultra-FGVC. By evaluating existing continual learning methods on the constructed continual ultra-FGVC benchmark, we observe that the main bottleneck lies in the limited model plasticity for incrementally adapting to new tasks. This can be caused by excessive anti-forgetting constraints as the difficult ultra-FGVC task requires substantial update of parameters, and over-fitting on early tasks given that the ultra-fine-grained categories are with very few training samples. To tackle these problems, we propose a joint self-supervised learning and prompting model. The prompt-based continual learning framework offers proper anti-forgetting operation by fixed pretrained vision transformer and adaptive prompt selection. By jointly optimizing the learnable prompts with an adversarial self-supervised loss, the over-fitting on each continual learning task is mitigated. Extensive experiments demonstrate that the proposed method outperforms existing continual learning methods on the challenging continual ultra-FGVC problem.

关键词：