检索结果-内蒙古大学图书馆

GM-DF: Generalized Multi-Scenario Deepfake Detection

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lai, Yingxin Yu, Zitong Yang, Jing Li, Bin Kang, Xiangui Shen, Linlin The School of Computing and Information Technology Great Bay University Dongguan523000 China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University Shenzhen518060 China Shenzhen Institute of Artificial Intelligence and Robotics for Society Shenzhen University Shenzhen518060 China The Guangdong Key Laboratory of Information Security The School of Computer Science and Engineering Sun Yat-sen University Guangzhou510080 China Computer Vision Institute School of Computer Science & Software Engineering Shenzhen Institute of Artificial Intelligence and Robotics for Society Guangdong Key Laboratory of Intelligent Information Processing National Engineering Laboratory for Big Data System Computing Technology Shenzhen University Shenzhen518060 China

Existing face forgery detection usually follows the paradigm of training models in a single domain, which leads to limited generalization capacity when unseen scenarios and unknown attacks occur. In this paper, we elaborately investigate the generalization capacity of deepfake detection models when jointly trained on multiple face forgery detection datasets. We first find a rapid degradation of detection accuracy when models are directly trained on combined datasets due to the discrepancy across collection scenarios and generation methods. To address the above issue, a Generalized Multi-Scenario Deepfake Detection framework (GM-DF) is proposed to serve multiple real-world scenarios by a unified model. First, we propose a hybrid expert modeling approach for domain-specific real/forgery feature extraction. Besides, as for the commonality representation, we use CLIP to extract the common features for better aligning visual and textual features across domains. Meanwhile, we introduce a masked image reconstruction mechanism to force models to capture rich forged details. Finally, we supervise the models via a domain-aware meta-learning strategy to further enhance their generalization capacities. Specifically, we design a novel domain alignment loss to strongly align the distributions of the meta-test domains and meta-train domains. Thus, the updated models are able to represent both specific and common real/forgery features across multiple datasets. In consideration of the lack of study of multi-dataset training, we establish a new benchmark leveraging multi-source data to fairly evaluate the models' generalization capacity on unseen scenarios. Both qualitative and quantitative experiments on five datasets conducted on traditional protocols as well as the proposed benchmark demonstrate the effectiveness of our approach. The codes will be available on https://***/laiyingxin2/GM-DF. Copyright © 2024, The Authors. All rights reserved.

关键词： Image reconstruction

Size-Invariance Matters: Rethinking Metrics and Losses for Imbalanced Multi-object Salient Object Detection 41

学校读者我要写书评

暂无评论

Size-Invariance Matters: Rethinking Metrics and Losses for I...

41st International Conference on Machine Learning, ICML 2024

作者： Li, Feiran Xu, Qianqian Bao, Shilong Yang, Zhiyong Cong, Runmin Cao, Xiaochun Huang, Qingming Institute of Information Engineering Chinese Academy of Sciences Beijing China School of Cyber Security University of Chinese Academy of Sciences Beijing China Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences Beijing China School of Computer Science and Technology University of Chinese Academy of Sciences Beijing China Institute of Information Science Beijing Jiaotong University Beijing China School of Control Science and Engineering Shandong University Jinan China Key Laboratory of Machine Intelligence and System Control Ministry of Education Jinan China School of Cyber Science and Tech. Sun Yat-sen University Shenzhen Campus China Key Laboratory of Big Data Mining and Knowledge Management Chinese Academy of Sciences Beijing China

This paper explores the size-invariance of evaluation metrics in Salient Object Detection (SOD), especially when multiple targets of diverse sizes co-exist in the same image. We observe that current metrics are size-sensitive, where larger objects are focused, and smaller ones tend to be ignored. We argue that the evaluation should be size-invariant because bias based on size is unjustified without additional semantic information. In pursuit of this, we propose a generic approach that evaluates each salient object separately and then combines the results, effectively alleviating the imbalance. We further develop an optimization framework tailored to this goal, achieving considerable improvements in detecting objects of different sizes. Theoretically, we provide evidence supporting the validity of our new metrics and present the generalization analysis of SOD. Extensive experiments demonstrate the effectiveness of our method. The code is available at https://***/Ferry-Li/SI-SOD. Copyright 2024 by the author(s)

关键词： Object detection

HairDiffusion: Vivid Multi-Colored Hair Editing via Latent Diffusion

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zeng, Yu Zhang, Yang Liu, Jiachen Shen, Linlin Deng, Kaijun He, Weizhao Wang, Jinbao Computer Vision Institute School of Computer Science & Software Engineering Shenzhen University China Shenzhen Institute of Artificial Intelligence and Robotics for Society China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University China Guangdong Provincial Key Laboratory of Intelligent Information Processing China

Hair editing is a critical image synthesis task that aims to edit hair color and hairstyle using text descriptions or reference images, while preserving irrelevant attributes (e.g., identity, background, cloth). Many existing methods are based on StyleGAN to address this task. However, due to the limited spatial distribution of StyleGAN, it struggles with multiple hair color editing and facial preservation. Considering the advancements in diffusion models, we utilize Latent Diffusion Models (LDMs) for hairstyle editing. Our approach introduces Multi-stage Hairstyle Blend (MHB), effectively separating control of hair color and hairstyle in diffusion latent space. Additionally, we train a warping module to align the hair color with the target region. To further enhance multi-color hairstyle editing, we fine-tuned a CLIP model using a multi-color hairstyle dataset. Our method not only tackles the complexity of multi-color hairstyles but also addresses the challenge of preserving original colors during diffusion editing. Extensive experiments showcase the superiority of our method in editing multi-color hairstyles while preserving facial attributes given textual descriptions and reference images. © 2024, CC BY.

关键词： Color image processing

DEGSTalk: Decomposed Per-Embedding Gaussian Fields for Hair-Preserving Talking Face Synthesis

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Deng, Kaijun Zheng, Dezhi Xie, Jindong Wang, Jinbao Xie, Weicheng Shen, Linlin Song, Siyang Computer Vision Institute School of Computer Science and Software Engineering Shenzhen University China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University China Guangdong Provincial Key Laboratory of Intelligent Information Processing China Department of Computer Science University of Exeter United Kingdom

Accurately synthesizing talking face videos and capturing fine facial features for individuals with long hair presents a significant challenge. To tackle these challenges in existing methods, we propose a decomposed per-embedding Gaussian fields (DEGSTalk), a 3D Gaussian Splatting (3DGS)-based talking face synthesis method for generating realistic talking faces with long hairs. Our DEGSTalk employs Deformable Pre-Embedding Gaussian Fields, which dynamically adjust pre-embedding Gaussian primitives using implicit expression coefficients. This enables precise capture of dynamic facial regions and subtle expressions. Additionally, we propose a Dynamic Hair-Preserving Portrait Rendering technique to enhance the realism of long hair motions in the synthesized videos. Results show that DEGSTalk achieves improved realism and synthesis quality compared to existing approaches, particularly in handling complex facial dynamics and hair preservation. Our code will be publicly available at https://***/CVI-SZU/DEGSTalk. Copyright © 2024, The Authors. All rights reserved.

关键词： Gaussian distribution

A virtualized data center energy-saving mechanism based on switching operating mode of physical servers and reserving virtual machines

学校读者我要写书评

暂无评论

A virtualized data center energy-saving mechanism based on s...

作者： Yin, Chunxia Liu, Jian Jin, Shunfu College of Information Science and Engineering Yanshan University Qinhuangdao China Key Laboratory for Computer Virtual Technology and System Integration of Hebei Province Yanshan University Qinhuangdao China Ocean College of Hebei Agricultural University Qinhuangdao China Institute of Computing Technology Chinese Academy of Sciences Beijing China

The energy consumption of virtualized data centers has grown very fast in last several years. Because a large number of hosts are running in an idle state, virtualized data centers waste a large amount of electric energy. To save more energy for virtualized data centers, an energy-saving mechanism is proposed based on switching operating mode of physical servers and reserving virtual machines (VMs). The main idea is that when the amount of idle VMs reaches twice of a specified threshold, half of these idle VMs are reserved to process the new task that is about to arrive, and the physical server which hosts the other half of the idle VMs is switched to the sleep mode. From the perspective of task arrival rate and sleep parameters, we use two-dimensional Markov processes to analyze the proposed energy-saving mechanisms. By using matrix geometry solutions, we theoretically estimate energy consumption and response performance. According to the numerical experiments, the proposed energy-saving mechanism obviously cut down electric energy consumption and ensures response performance. Finally, the specified threshold for the amount of VMs is optimized by building a cost function. © 2020 John Wiley & Sons, Ltd.

关键词： Cost functions

SFDA-rPPG: Source-Free Domain Adaptive Remote Physiological Measurement with Spatio-Temporal Consistency

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Xie, Yiping Yu, Zitong Wu, Bingjie Xie, Weicheng Shen, Linlin Computer Vision Institute School of Computer Science & Software Engineering Shenzhen Institute of Artificial Intelligence and Robotics for Society Guangdong Key Laboratory of Intelligent Information Processing Shenzhen University Shenzhen518060 China School of Computing and Information Technology Great Bay University Dongguan523000 China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University Shenzhen518060 China Singapore

Remote Photoplethysmography (rPPG) is a non-contact method that uses facial video to predict changes in blood volume, enabling physiological metrics measurement. Traditional rPPG models often struggle with poor generalization capacity in unseen domains. Current solutions to this problem is to improve its generalization in the target domain through Domain Generalization (DG) or Domain Adaptation (DA). However, both traditional methods require access to both source domain data and target domain data, which cannot be implemented in scenarios with limited access to source data, and another issue is the privacy of accessing source domain data. In this paper, we propose the first Source-free Domain Adaptation benchmark for rPPG measurement (SFDA-rPPG), which overcomes these limitations by enabling effective domain adaptation without access to source domain data. Our framework incorporates a Three-Branch Spatio-Temporal Consistency Network (TSTC-Net) to enhance feature consistency across domains. Furthermore, we propose a new rPPG distribution alignment loss based on the Frequency-domain Wasserstein Distance (FWD), which leverages optimal transport to align power spectrum distributions across domains effectively and further enforces the alignment of the three branches. Extensive cross-domain experiments and ablation studies demonstrate the effectiveness of our proposed method in source-free domain adaptation settings. Our findings highlight the significant contribution of the proposed FWD loss for distributional alignment, providing a valuable reference for future research and applications. The source code is available at https://***/XieYiping66/SFDA-rPPG. Copyright © 2024, The Authors. All rights reserved.

关键词： Photoplethysmography

FIRE: A Dataset for Feedback Integration and Refinement Evaluation of Multimodal Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Pengxiang Gao, Zhi Zhang, Bofei Yuan, Tao Wu, Yuwei Harandi, Mehrtash Jia, Yunde Zhu, Song-Chun Li, Qing Beijing Key Laboratory of Intelligent Information Technology School of Computer Science & Technology Beijing Institute of Technology China State Key Laboratory of General Artificial Intelligence BIGAI China State Key Laboratory of General Artificial Intelligence Peking University China Department of Electrical and Computer System Engineering Monash University Australia Guangdong Laboratory of Machine Perception and Intelligent Computing Shenzhen MSU-BIT University China Department of Automation Tsinghua University China

Vision language models (VLMs) have achieved impressive progress in diverse applications, becoming a prevalent research direction. In this paper, we build FIRE, a feedback-refinement dataset, consisting of 1.1M multi-turn conversations that are derived from 27 source datasets, empowering VLMs to spontaneously refine their responses based on user feedback across diverse tasks. To scale up the data collection, FIRE is collected in two components: FIRE-100K and FIRE-1M, where FIRE-100K is generated by GPT-4V, and FIRE-1M is freely generated via models trained on FIRE-100K. Then, we build FIRE-Bench, a benchmark to comprehensively evaluate the feedback-refining capability of VLMs, which contains 11K feedback-refinement conversations as the test data, two evaluation settings, and a model to provide feedback for VLMs. We develop the FIRE-LLaVA model by fine-tuning LLaVA on FIRE-100K and FIRE-1M, which shows remarkable feedback-refining capability on FIRE-Bench and outperforms untrained VLMs by 50%, making more efficient user-agent interactions and underscoring the significance of the FIRE dataset. Copyright © 2024, The Authors. All rights reserved.

关键词： Fireclay

FedPHE: A Secure and Efficient Federated Learning via Packed Homomorphic Encryption

学校读者我要写书评

暂无评论

IEEE Transactions on Dependable and Secure computing 2025年

作者： Li, Yuqing Yan, Nan Chen, Jing Wang, Xiong Hong, Jianan He, Kun Wang, Wei Li, Bo Wuhan University Key Laboratory of Aerospace Information Security and Trusted Computing Ministry of Education School of Cyber Science and Engineering Wuhan430072 China Wuhan University RiZhao Information Technology Institute Rizhao276800 China Huazhong University of Science and Technology National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab/Cluster and Grid Computing Lab School of Computer Science and Technology Wuhan430074 China Shanghai Jiao Tong University School of Cyber Science and Engineering Shanghai200240 China Hong Kong University of Science and Technology Department of Computer Science and Engineering Hong Kong

Cross-silo federated learning (FL) enables multiple institutions (clients) to collaboratively build a global model without sharing private data. To prevent privacy leakage during aggregation, homomorphic encryption (HE) is widely used to encrypt model updates, yet incurs high computation and communication overheads. To reduce these overheads, packed HE (PHE) has been proposed to encrypt multiple plaintexts into a single ciphertext. However, the original design of PHE assumes all clients share a single private key, making the system vulnerable to security threats of ciphertexts being intercepted and decrypted by honest-but-curious clients. Also, it does not consider the heterogeneity among different clients, resulting in undermined training efficiency with slow convergence and stragglers. To address these challenges, we propose FedPHE, a secure and efficient FL framework with PHE by jointly exploiting contribution-aware secure aggregation and straggler-resistant client selection. Using CKKS with sparsification and blinding, FedPHE achieves efficient secure aggregation that allows clients to only provide obscured encrypted updates while the server can perform aggregation by accounting for contributions of local updates. To mitigate the straggler effect, we devise a perturbed sketch-based selection to cherry-pick representative clients with heterogeneous models and computing capabilities in a communication-efficient and privacy-preserving manner. We show, through rigorous security analysis and extensive experiments, that FedPHE can efficiently safeguard clients' privacy, achieve 2.45-6.56× training speedup, cut the communication overhead by 1.32-24.85×, and reduce straggler effects by 1.89-2.78×. © 2004-2012 IEEE.

关键词： Privacy by design

TDiffSal: Text-Guided Diffusion Saliency Prediction Model for Images 27th

学校读者我要写书评

暂无评论

TDiffSal: Text-Guided Diffusion Saliency Prediction Model fo...

27th International Conference on Pattern Recognition, ICPR 2024

作者： Zhang, Nana Xiong, Min Zhu, Dandan Zhu, Kun Zhai, Guangtao School of Computer Science and Technology Donghua University No. 2999 Renmin North Road Songjiang District Shanghai201620 China State Key Laboratory for Novel Software Technology Nanjing University Nanjing210093 China Institute of AI Education Shanghai East China Normal University No. 3663 Zhongshan North Road Putuo District Shanghai200333 China Key Laboratory of Embedded System and Service Computing Ministry of Education Tongji University No. 4800 Cao’an Highway Jiading District Shanghai201804 China Institute of Image Communication and Network Engineering Shanghai Jiao Tong University No. 800 Dongchuan Road Minhang District Shanghai200240 China

ISBN: (纸本)9783031781858

Existing visual saliency prediction methods mainly focus on single-modal visual saliency prediction, while ignoring the significant impact of text on visual saliency. To more comprehensively explore the influence of text on human attention in images, we propose a text-guided diffusion saliency prediction model, named TDiffSal. In specific, recent studies on stable diffusion models have shown promising performance in unifying tasks due to their inherent generalization ability. Inspired by this, a novel diffusion model for generalized visual-text saliency prediction is proposed, which formulates the prediction issue as a conditional generative task of the saliency map by employing input visual and text as the conditions. Meanwhile, we introduce a multi-head fusion module to effectively integrate text features and image features, which can efficiently guide the image denoising process and progressively refine the generated saliency map to make it semantically relevant to the text. Additionally, we employ an efficient pre-training strategy to enhance the robustness and generalization of the proposed model. We conduct extensive experiments on benchmark datasets to demonstrate its superior performance compared to other state-of-the-art methods. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

关键词： Prediction models