检索结果-内蒙古大学图书馆

Talking Head Generation for Media Interaction System with Feature Disentanglement

学校读者我要写书评

暂无评论

Talking Head Generation for Media Interaction System with Fe...

International Conference on Parallel and Distributed Systems (ICPADS)

作者： Lei Zhang Qingyang Chen Zhilei Liu College of Intelligence and Computing School of New Media and Communication Tianjin University Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University Changchun P.R. China State Key Laboratory for Novel Software Technology Nanjing University P.R. China School of New Media and Communication Tianjin University Tianjin China College of Intelligence and Computing Tianjin University Tianjin China

The task of talking head generation for the media interaction system is to take images and audio clips of the target face as input, and generate a realistic video of the target synchronized with the audio. Most of the existing works directly take all the information in the image as input, which causes the problem of feature information redundancy. At the same time, in the stage of feature fusion, the image and audio information is directly spliced, and the relationship between the two modal information is ignored. To solve the problems, we initially implement the disentanglement of image by introducing a loss function to separate the image into identity features and content-related features. Besides, we introduce a multi-head selfattention mechanism to learn the relationship between the two modal information of image and audio, and implement the full fusion of multi-modal information. In addition, we validate the effectiveness of our model through extensive quantitative and qualitative analysis of two datasets. Extensive experiments show the superiority of the proposed model in all aspects.

关键词： Analytical models Redundancy Media Synchronization Task analysis Faces

Multi-focus image fusion based on fully convolutional networks

学校读者我要写书评

暂无评论

Frontiers of Information Technology & Electronic engineering 2020年第7期21卷 1019-1033页

作者： Rui GUO Xuan-jing SHEN Xiao-yu DONG Xiao-li ZHANG Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin UniversityChangchun 130012China College of Computer Science and Technology Jilin UniversityChangchun 130012China

We propose a multi-focus image fusion method, in which a fully convolutional network for focus detection(FD-FCN) is constructed. To obtain more precise focus detection maps, we propose to add skip layers in the network to make both detailed and abstract visual information available when using FD-FCN to generate maps. A new training dataset for the proposed network is constructed based on dataset CIFAR-10. The image fusion algorithm using FD-FCN contains three steps: focus maps are obtained using FD-FCN, decision map generation occurs by applying a morphological process on the focus maps, and image fusion occurs using a decision map. We carry out several sets of experiments, and both subjective and objective assessments demonstrate the superiority of the proposed fusion method to state-of-the-art algorithms.

关键词： Multi-focus image fusion Fully convolutional networks Skip layer Performance evaluation

Bounding-box deep calibration for high performance face detection

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Luo, Shi Li, Xiongfei Zhang, Xiaoli College of Computer Science and Technology Jilin University Changchun130012 China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University Changchun China

Modern convolutional neural networks (CNNs)‐based face detectors have achieved tremendous strides due to large annotated datasets. However, misaligned results with high detection confidence but low localization accuracy restrict the further improvement of detection performance. In this paper, the authors first predict high confidence detection results on the training set itself. Surprisingly, a considerable part of them exist in the same misalignment problem. Then, the authors carefully examine these cases and point out that annotation misalignment is the main reason. Later, a comprehensive discussion is given for the replacement rationality between predicted and annotated bounding‐boxes. Finally, the authors propose a novel Bounding‐Box Deep Calibration (BDC) method to reasonably replace misaligned annotations with model predicted bounding‐boxes and offer calibrated annotations for the training set. Extensive experiments on multiple detectors and two popular benchmark datasets show the effectiveness of BDC on improving models' precision and recall rate, without adding extra inference time and memory consumption. Our simple and effective method provides a general strategy for improving face detection, especially for light‐weight detectors in real‐time situations. © 2021, CC BY.

关键词： Face recognition

Recovering accurate labeling information from partially valid data for effective multi-label learning 29

学校读者我要写书评

暂无评论

Recovering accurate labeling information from partially vali...

29th International Joint Conference on Artificial Intelligence, IJCAI 2020

作者： Li, Ximing Wang, Yang College of Computer Science and Technology Jilin University China Key Laboratory of Symbolic Computation Knowledge Engineering of Ministry of Education Jilin University China Key Laboratory of Knowledge Engineering With Big Data Ministry of Education Hefei University of Technology China School of Computer Sci and Information Engineering Hefei University of Technology China

ISBN: (纸本)9780999241165

Partial Multi-label Learning (PML) aims to induce the multi-label predictor from datasets with noisy supervision, where each training instance is associated with several candidate labels but only partially valid. To address the noisy issue, the existing PML methods basically recover the ground-truth labels by leveraging the ground-truth confidence of the candidate label, i.e., the likelihood of a candidate label being a ground-truth one. However, they neglect the information from non-candidate labels, which potentially contributes to the ground-truth label recovery. In this paper, we propose to recover the ground-truth labels, i.e., estimating the ground-truth confidences, from the label enrichment, composed of the relevance degrees of candidate labels and irrelevance degrees of non-candidate labels. Upon this observation, we further develop a novel two-stage PML method, namely Partial MultiLabel Learning with Label Enrichment-Recovery (PML3ER), where in the first stage, it estimates the label enrichment with unconstrained label propagation, then jointly learns the ground-truth confidence and multi-label predictor given the label enrichment. Experimental results validate that PML3ER outperforms the state-of-the-art PML methods. © 2020 Inst. Sci. inf., Univ. Defence in Belgrade. All rights reserved.

关键词： Learning systems

Aerial Reliable Collaborative Communications for Terrestrial Mobile Users via Evolutionary Multi-Objective Deep Reinforcement Learning

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Sun, Geng Xiao, Jian Li, Jiahui Wang, Jiacheng Kang, Jiawen Niyato, Dusit Mao, Shiwen College of Computer Science and Technology Jilin University Changchun130012 China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University Changchun130012 China affiliated with the College of Computing and Data Science Nanyang Technological University Singapore639798 Singapore School of Computer Science and Engineering Nanyang Technological University Singapore639798 Singapore School of Automation Guangdong University of Technology Guangzhou510641 China Department of Electrical and Computer Engineering Auburn University AuburnAL36849-5201 United States

Unmanned aerial vehicles (UAVs) have emerged as the potential aerial base stations (BSs) to improve terrestrial communications. However, the limited onboard energy and antenna power of a UAV restrict its communication range and transmission capability. To address these limitations, this work employs collaborative beamforming through a UAV-enabled virtual antenna array to improve transmission performance from the UAV to terrestrial mobile users, under interference from non-associated BSs and dynamic channel conditions. Specifically, we introduce a memory-based random walk model to more accurately depict the mobility patterns of terrestrial mobile users. Following this, we formulate a multi-objective optimization problem (MOP) focused on maximizing the transmission rate while minimizing the flight energy consumption of the UAV swarm. Given the NP-hard nature of the formulated MOP and the highly dynamic environment, we transform this problem into a multi-objective Markov decision process and propose an improved evolutionary multi-objective reinforcement learning algorithm. Specifically, this algorithm introduces an evolutionary learning approach to obtain the approximate Pareto set for the formulated MOP. Moreover, the algorithm incorporates a long short-term memory network and hyper-sphere-based task selection method to discern the movement patterns of terrestrial mobile users and improve the diversity of the obtained Pareto set. Simulation results demonstrate that the proposed method effectively generates a diverse range of non-dominated policies and outperforms existing methods. Additional simulations demonstrate the scalability and robustness of the proposed CB-based method under different system parameters and various unexpected circumstances. © 2025, CC0.

关键词： Unmanned aerial vehicles (UAV)

Lightweight Deep Learning for Resource-Constrained Environments: A Survey

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Liu, Hou-I Galindo, Marco Xie, Hongxia Wong, Lai-Kuan Shuai, Hong-Han Li, Yung-Yui Cheng, Wen-Huang Department of Electronics and Electrical Engineering National Yang Ming Chiao Tung University Hsinchu300 Taiwan College of Computer Science and Technology Jilin University Key Laboratory of Symbolic Computation and Knowledge Engineering Ministry of Education Jilin University Changchun130000 China Faculty of Computing and Informatics Multimedia University Cyberjaya63100 Malaysia Hon Hai Research Institute Taipei114 Taiwan Department of Computer Science and Information Engineering National Taiwan University Taipei106 Taiwan

Over the past decade, the dominance of deep learning has prevailed across various domains of artificial intelligence, including natural language processing, computer vision, and biomedical signal processing. While there have been remarkable improvements in model accuracy, deploying these models on lightweight devices, such as mobile phones and microcontrollers, is constrained by limited resources. In this survey, we provide comprehensive design guidance tailored for these devices, detailing the meticulous design of lightweight models, compression methods, and hardware acceleration strategies. The principal goal of this work is to explore methods and concepts for getting around hardware constraints without compromising the model's accuracy. Additionally, we explore two notable paths for lightweight deep learning in the future: deployment techniques for TinyML and Large Language Models. Although these paths undoubtedly have potential, they also present significant challenges, encouraging research into unexplored areas. Copyright © 2024, The Authors. All rights reserved.

关键词： Computer hardware

Document-Level Relation Extraction with Entity Type Constraints

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Han, Ridong Peng, Tao Zhu, BeiBei Bi, Haijia Han, Jiayu Liu, Lu College of Computer Science and Technology Jilin University Jilin Changchun130012 China College of Software Jilin University Jilin Changchun130012 China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin Changchun130012 China College of Computer and Artificial Intelligence Liaoning Normal University Liaoning Dalian116081 China Department of Linguistics University of Washington SeattleWA98195 United States

Long-tail problem and multi-label problem are two commonly encountered challenges in document-level relation extraction task. Current efforts are concerned with enhancing the representations of entity pairs through Transformer or document graphs, which cannot tackle the above challenges well. Relation correlations are a potential solution, which allows head relations to assist in the training of tail ones by transferring correlation knowledge between them, and can measure the semantic distance between relations to assist the classifier in assigning multiple semantically similar relations to multi-label instances. This paper proposes to learn relation correlations from both global and local views using entity type constraints, which means that the subject-object entity types limit the scope of possible relations. Globally, we statistically construct the Type-constrained Graph between entity types and relations, which formulates all possible subject/object types for each relation. Different relations are connected by common entity types, reflecting the desired correlations. Locally, given an entity pair, the classification probability of relations matching its entity types should be greater than those unmatched. Therefore, the Type-constrained Loss is proposed to make the matched relations have greater probabilities. Detailed experiments are conducted on DocRED and DWIE datasets, and our model significantly outperforms baselines under long-tail and multi-label setups. © 2024, The Authors. All rights reserved.

关键词： Extraction

PepLand: A large-scale pre-trained peptide representation model for a comprehensive landscape of both canonical and non-canonical amino acids

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhang, Ruochi Wu, Haoran Xiu, Yuting Li, Kewei Chen, Ningning Wang, Yu Wang, Yan Gao, Xin Zhou, Fengfeng Key Laboratory of Symbolic Computation and Knowledge Engineering Ministry of Education Jilin University Jilin Changchun130012 China School of Artificial Intelligence Jilin University Changchun130012 China Syneron Technology Guangzhou510700 China College of Computer Science and Technology Jilin University Jilin Changchun130012 China Thuwal23955 Saudi Arabia Thuwal23955 Saudi Arabia

In recent years, the scientific community has become increasingly interested on peptides with non-canonical amino acids due to their superior stability and resistance to proteolytic degradation. These peptides present promising modifications to biological, pharmacological, and physiochemical attributes in both endogenous and engineered peptides. Notwithstanding their considerable advantages, the scientific community exhibits a conspicuous absence of an effective pre-trained model adept at distilling feature representations from such complex peptide sequences. We herein propose PepLand, a novel pre-training architecture for representation and property analysis of peptides spanning both canonical and non-canonical amino acids. In essence, PepLand leverages a comprehensive multi-view heterogeneous graph neural network tailored to unveil the subtle structural representations of peptides. Empirical validations underscore PepLand's effectiveness across an array of peptide property predictions, encompassing protein-protein interactions, permeability, solubility, and synthesizability. The rigorous evaluation confirms PepLand's unparalleled capability in capturing salient synthetic peptide features, thereby laying a robust foundation for transformative advances in peptide-centric research domains. We have made all the source code utilized in this study publicly accessible via GitHub at https://***/zhangruochi/pepland. © 2023, CC BY-NC-ND.

关键词： Peptides

When Federated Recommendation Meets Cold-Start Problem: Separating Item Attributes and User Interactions

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhang, Chunxu Zhang, Zijian Long, Guodong Yan, Peng Zhou, Tianyi Yang, Bo College of Computer Science and Technology Jilin University Key Laboratory of Symbolic Computation and Knowledge Engineering Ministry of Education Changchun China College of Computer Science and Technology Jilin University City University of Hong Kong China Australian Artificial Intelligence Institute FEIT University of Technology Sydney Sydney Australia Computer Science and UMIACS University of Maryland Maryland United States

Federated recommendation systems usually trains a global model on the server without direct access to users’ private data on their own devices. However, this separation of the recommendation model and users’ private data poses a challenge in providing quality service, particularly when it comes to new items, namely cold-start recommendations in federated settings. This paper introduces a novel method called Item-aligned Federated Aggregation (IFedRec) to address this challenge. It is the first research work in federated recommendation to specifically study the cold-start scenario. The proposed method learns two sets of item representations by leveraging item attributes and interaction records simultaneously. Additionally, an item representation alignment mechanism is designed to align two item representations and learn the meta attribute network at the server within a federated learning framework. Experiments on four benchmark datasets demonstrate IFedRec’s superior performance for cold-start scenarios. Furthermore, we also verify IFedRec owns good robustness when the system faces limited client participation and noise injection, which brings promising practical application potential in privacy-protection enhanced federated recommendation systems. The implementation code is available1 Copyright © 2023, The Authors. All rights reserved.

关键词： Recommender systems