检索结果-内蒙古大学图书馆

MFCLIP: Multi-modal Fine-grained CLIP for Generalizable Diffusion Face Forgery Detection

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Yaning Wang, Tianyi Yu, Zitong Gao, Zan Shen, Linlin Chen, Shengyong Jinan250353 China Nanyang Technological University 50 Nanyang Ave Block N 4 639798 Singapore School of Computing and Information Technology Great Bay University Dongguan523000 China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University Shenzhen518060 China Jinan250014 China The Key Laboratory of Computer Vision and System Ministry of Education Tianjin University of Technology Tianjin300384 China Computer Vision Institute College of Computer Science and Software Engineering Shenzhen University Shenzhen518060 China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University Shenzhen China Shenzhen Institute of Artificial Intelligence and Robotics for Society Shenzhen518129 China Guangdong Key Laboratory of Intelligent Information Processing Shenzhen University Shenzhen518060 China

The rapid development of photo-realistic face generation methods has raised significant concerns in society and academia, highlighting the urgent need for robust and generalizable face forgery detection (FFD) techniques. Although existing approaches mainly capture face forgery patterns using image modality, other modalities like fine-grained noises and texts are not fully explored, which limits the generalization capability of the model. In addition, most FFD methods tend to identify facial images generated by GAN, but struggle to detect unseen diffusion-synthesized ones. To address the limitations, we aim to leverage the cutting-edge foundation model, contrastive language-image pre-training (CLIP), to achieve generalizable diffusion face forgery detection (DFFD). In this paper, we propose a novel multi-modal fine-grained CLIP (MFCLIP) model, which mines comprehensive and fine-grained forgery traces across image-noise modalities via language-guided face forgery representation learning, to facilitate the advancement of DFFD. Specifically, we devise a fine-grained language encoder (FLE) that extracts fine global language features from hierarchical text prompts. We design a multi-modal vision encoder (MVE) to capture global image forgery embeddings as well as fine-grained noise forgery patterns extracted from the richest patch, and integrate them to mine general visual forgery traces. Moreover, we build an innovative plug-and-play sample pair attention (SPA) method to emphasize relevant negative pairs and suppress irrelevant ones, allowing cross-modality sample pairs to conduct more flexible alignment. Extensive experiments and visualizations show that our model outperforms the state of the arts on different settings like cross-generator, cross-forgery, and cross-dataset evaluations. Copyright © 2024, The Authors. All rights reserved.

关键词： Alignment

Contrastive neural architecture search with neural architecture comparators

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Chen, Yaofo Guo, Yong Chen, Qi Li, Minli Zeng, Wei Wang, Yaowei Tan, Mingkui South China University of Technology China Peng Cheng Laboratory China Peking University China Key Laboratory of Big Data and Intelligent Robot Ministry of Education China

One of the key steps in Neural Architecture Search (NAS) is to estimate the performance of candidate architectures. Existing methods either directly use the validation performance or learn a predictor to estimate the performance. However, these methods can be either computationally expensive or very inaccurate, which may severely affect the search efficiency and performance. Moreover, as it is very difficult to annotate architectures with accurate performance on specific tasks, learning a promising performance predictor is often non-trivial due to the lack of labeled data. In this paper, we argue that it may not be necessary to estimate the absolute performance for NAS. On the contrary, we may need only to understand whether an architecture is better than a baseline one. However, how to exploit this comparison information as the reward and how to well use the limited labeled data remains two great challenges. In this paper, we propose a novel Contrastive Neural Architecture Search (CTNAS) method which performs architecture search by taking the comparison results between architectures as the reward. Specifically, we design and learn a Neural Architecture Comparator (NAC) to compute the probability of candidate architectures being better than a baseline one. Moreover, we present a baseline updating scheme to improve the baseline iteratively in a curriculum learning manner. More critically, we theoretically show that learning NAC is equivalent to optimizing the ranking over architectures. Extensive experiments in three search spaces demonstrate the superiority of our CTNAS over existing methods. Copyright © 2021, The Authors. All rights reserved.

关键词： Comparators (optical)

Hard Sample Matters a Lot in Zero-Shot Quantization

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Li, Huantong Wu, Xiangmiao Lv, Fanbing Liao, Daihai Li, Thomas H. Zhang, Yonggang Han, Bo Tan, Mingkui South China University of Technology China Changsha Hisense Intelligent System Research Institute Co. Ltd China Hong Kong Baptist University Hong Kong Key Laboratory of Big Data and Intelligent Robot Ministry of Education China PengCheng Laboratory China Information Technology R&D Innovation Center of Peking University China School of Electronic and Computer Engineering Peking University Shenzhen Graduate School Shenzhen China

Zero-shot quantization (ZSQ) is promising for compressing and accelerating deep neural networks when the data for training full-precision models are inaccessible. In ZSQ, network quantization is performed using synthetic samples, thus, the performance of quantized models depends heavily on the quality of synthetic samples. Nonetheless, we find that the synthetic samples constructed in existing ZSQ methods can be easily fitted by models. Accordingly, quantized models obtained by these methods suffer from significant performance degradation on hard samples. To address this issue, we propose HArd sample Synthesizing and Training (HAST). Specifically, HAST pays more attention to hard samples when synthesizing samples and makes synthetic samples hard to fit when training quantized models. HAST aligns features extracted by full-precision and quantized models to ensure the similarity between features extracted by these two models. Extensive experiments show that HAST significantly outperforms existing ZSQ methods, achieving performance comparable to models that are quantized with real data. Copyright © 2023, The Authors. All rights reserved.

关键词： Deep neural networks

Directed Acyclic Graph Factorization Machines for CTR Prediction via Knowledge Distillation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Tian, Zhen Bai, Ting Zhang, Zibin Xu, Zhiyuan Lin, Kangyi Wen, Ji-Rong Zhao, Wayne Xin Gaoling School of Artificial Intelligence Renmin University of China Beijing China Beijing University of Posts and Telecommunications Beijing China Weixin Open Platform Tencent Guangzhou China Beijing Key Laboratory of Intelligent Telecommunications Software and Multimedia China Beijing Key Laboratory of Big Data Management and Analysis Methods China

With the growth of high-dimensional sparse data in web-scale recommender systems, the computational cost to learn high-order feature interaction in CTR prediction task largely increases, which limits the use of high-order interaction models in real industrial applications. Some recent knowledge distillation based methods transfer knowledge from complex teacher models to shallow student models for accelerating the online model inference. However, they suffer from the degradation of model accuracy in knowledge distillation process. It is challenging to balance the efficiency and effectiveness of the shallow student models. To address this problem, we propose a Directed Acyclic Graph Factorization Machine (KD-DAGFM) to learn the high-order feature interactions from existing complex interaction models for CTR prediction via Knowledge Distillation. The proposed lightweight student model DAGFM can learn arbitrary explicit feature interactions from teacher networks, which achieves approximately lossless performance and is proved by a dynamic programming algorithm. Besides, an improved general model KD-DAGFM+ is shown to be effective in distilling both explicit and implicit feature interactions from any complex teacher model. Extensive experiments are conducted on four real-world datasets, including a large-scale industrial dataset from WeChat platform with billions of feature dimensions. KD-DAGFM achieves the best performance with less than 21.5% FLOPs of the state-ofthe-art method on both online and offline experiments, showing the superiority of DAGFM to deal with the industrial scale data in CTR prediction task1 © 2022, CC BY.

关键词： Recommender systems

Multi-scale Adaptive Semantic Segmentation Network for Remote Sensing Images based on Attention Gating Mechanism

学校读者我要写书评

暂无评论

Multi-scale Adaptive Semantic Segmentation Network for Remot...

Advanced Electronic Materials, Computers and Software Engineering (AEMCSE), International Conference on

作者： Yiguo Lu Yurong Qian Lu Bai Peng Liu Yinyin Fan Weijun Gong Key Laboratory of Software Engineering Xinjiang University Urumqi Xinjiang China Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing Xinjiang University Urumqi Xinjiang China School of Computer Science and Technology Xinjiang University Urumqi Xinjiang China Xinjiang Engineering Research Center of Big Data and Intelligent Software School of Software Xinjiang University Urumqi Xinjiang China School of Information Technology and Communication Hexi University Zhangye Gansu China

ISBN: (数字)9798331510916

ISBN: (纸本)9798331510923

In this paper, a multi-scale adaptive semantic segmentation model for remote sensing images—AG-MSASNet—is proposed to address the imprecise utilization of feature information in multi-scale feature extraction. The model introduces a novel attention-gated multi-scale feature fusion mechanism, which selectively integrates multi-scale features to enhance useful information while suppressing noise, thereby improving segmentation accuracy. To further enhance semantic understanding in complex remote sensing scenes, a Global Context Block (GC Block) is designed to capture long-range dependencies and aggregate global context. Additionally, a joint loss optimization strategy is proposed to balance learning across different scales and classes, contributing to more precise and robust segmentation. Experimental results on the LoveDA dataset demonstrate that the proposed model outperforms the baseline, achieving a $\mathbf{2 . 2 \%}$ improvement in mean intersection over union (mIoU) and a $4.1 \%$ increase in overall accuracy (OA).

关键词： Adaptation models Accuracy Attention mechanisms Adaptive systems Sensitivity Semantic segmentation Semantics Feature extraction Remote sensing Software engineering

Privacy-Preserved Neural Graph Similarity Learning

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Hou, Yupeng Zhao, Wayne Xin Li, Yaliang Wen, Ji-Rong Gaoling School of Artificial Intelligence Renmin University of China China Alibaba Group China Beijing Key Laboratory of Big Data Management and Analysis Methods China Engineering Research Center of Next-Generation Intelligent Search and Recommendation Ministry of Education

To develop effective and efficient graph similarity learning (GSL) models, a series of data-driven neural algorithms have been proposed in recent years. Although GSL models are frequently deployed in privacy-sensitive scenarios, the user privacy protection of neural GSL models has not drawn much attention. To comprehensively understand the privacy protection issues, we first introduce the concept of attackable representation to systematically characterize the privacy attacks that each model can face. Inspired by the qualitative results, we propose a novel Privacy-Preserving neural Graph Matching network model, named PPGM, for graph similarity learning. To prevent reconstruction attacks, the proposed model does not communicate node-level representations between devices. Instead, we learn multi-perspective graph representations based on learnable context vectors. To alleviate the attacks to graph properties, the obfuscated features that contain information from both graphs are communicated. In this way, the private properties of each graph can be difficult to infer. Based on the node-graph matching techniques while calculating the obfuscated features, PPGM can also be effective in similarity measuring. To quantitatively evaluate the privacy-preserving ability of neural GSL models, we further propose an evaluation protocol via training supervised black-box attack models. Extensive experiments on widely-used benchmarks show the effectiveness and strong privacy-protection ability of the proposed model PPGM. The code is available at: https://***/RUCAIBox/PPGM. © 2022, CC BY.

关键词： Graph neural networks

FaceBench: A Multi-View Multi-Level Facial Attribute VQA dataset for Benchmarking Face Perception MLLMs

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Wang, Xiaoqin Ma, Xusen Hou, Xianxu Ding, Meidan Li, Yudong Chen, Junliang Chen, Wenting Peng, Xiaoyang Shen, Linlin Computer Vision Institute College of Computer Science and Software Engineering Shenzhen University China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University China Guangdong Provincial Key Laboratory of Intelligent Information Processing China AIAC Xi’an Jiaotong-Liverpool University China Tsinghua University China The Hong Kong Polytechnic University Hong Kong City University of Hong Kong Hong Kong Sun Yat-sen University China

Multimodal large language models (MLLMs) have demonstrated remarkable capabilities in various tasks. However, effectively evaluating these MLLMs on face perception remains largely unexplored. To address this gap, we introduce FaceBench, a dataset featuring hierarchical multi-view and multi-level attributes specifically designed to assess the comprehensive face perception abilities of MLLMs. Initially, we construct a hierarchical facial attribute structure, which encompasses five views with up to three levels of attributes, totaling over 210 attributes and 700 attribute values. Based on the structure, the proposed FaceBench consists of 49,919 visual question-answering (VQA) pairs for evaluation and 23,841 pairs for fine-tuning. Moreover, we further develop a robust face perception MLLM baseline, Face-LLaVA, by training with our proposed face VQA data. Extensive experiments on various mainstream MLLMs and Face-LLaVA are conducted to test their face perception ability, with results also compared against human performance. The results reveal that, the existing MLLMs are far from satisfactory in understanding the fine-grained facial attributes, while our Face-LLaVA significantly outperforms existing open-source models with a small amount of training data and is comparable to commercial ones like GPT-4o and Gemini. The dataset will be released at https://***/CVI-SZU/FaceBench. Copyright © 2025, The Authors. All rights reserved.

关键词：

Multi User Offloading Strategy for Edge Computing in Charging Load Forecasting Scenario

学校读者我要写书评

暂无评论

Multi User Offloading Strategy for Edge Computing in Chargin...

Chinese Automation Congress (CAC)

作者： Hongyu Long Zhaocheng Huang Chao Liu Daolin Xu Tao Li Fan Ye Key Laboratory of Big Data Intelligent Computing School of Automation/School of Industrial Internet Chongqing University of Posts and Telecommunications Chongqing China School of Automation/School of Industrial Internet Chongqing University of Posts and Telecommunications Chongqing China Economic and Technical Research Institute of State Grid Chongqing Electric Power Company Chongqing China

In order to mitigate the impact of large-scale new energy vehicle charging access to the grid, edge computing technology is widely used in smart charging perception scenarios. By deploying various edge terminal devices or application software at the perception sites, it can collect and predict the power load in various regions, so as to effectively guide and control the charging amount of new energy vehicles. However, with the increase of computing demand brought by the increase of edge device nodes, the edge server may face the problem of insufficient computing resources, which will affect the low latency characteristics. To alleviate this problem, this paper abstracts each edge terminal device as an independent user with task computing needs, establishes a multi-user edge computing system, and proposes a multi-level resource allocation strategy for edge computing, Decompose and allocate communication channel resources and server resources in multiple rounds. This method can reduce task latency by optimizing the task offloading ratio and resource allocation method of users while meeting their basic needs. The simulation results show that this method can effectively reduce the latency of task computation in multi user scenarios.

关键词：

a branch-and-price approach for the nurse rostering problem with multiple units

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Hu, Wanzhe He, Xiaozhou Luo, Li Pardalos, Panos M. School of Economics and Management Chongqing University of Posts and Telecommunications Chongqing China Business School Sichuan University Sichuan Chengdu China Key Laboratory of Big Data Intelligent Computing Chongqing University of Posts and Telecommunications Chongqing China Department of Industrial and Systems Engineering Center for Applied Optimization University of Florida Gainesville United States

In this paper, we study the nurse rostering problem that considers multiple units and many soft time-related constraints. An efficient branch and price solution approach that relies on a fast algorithm to solve the pricing subproblem of the column generation process is presented. For the nurse rostering problem, its pricing subproblem can be formulated as a shortest path problem with resource constraints, which has been the backbone of several solutions for several classical problems like vehicle routing problems. However, approaches that perform well on these problems cannot be used since most constraints in the nurse rostering problem are soft. Based on ideas borrowed from global constraints in constraint programming to model rostering problems, an efficient dynamic programming algorithm with novel label definitions and dominating rules specific to soft time-related constraints is proposed. In addition, several acceleration strategies are employed to improve the branch and price algorithm. Computational results on instances of different sizes indicate that the proposed algorithm is a promising solution for the nurse rostering problem with multiple units. Copyright © 2023, The Authors. All rights reserved.

关键词： Constraint programming