检索结果-内蒙古大学图书馆

INITNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Guo, Xiefan Liu, Jinlin Cui, Miaomiao Li, Jiankai Yang, Hongyu Huang, Di State Key Laboratory of Software Development Environment Beihang University Beijing China School of Computer Science and Engineering Beihang University Beijing China Institute of Artificial Intelligence Beihang University Beijing China Shanghai Artificial Intelligence Laboratory Shanghai China

Recent strides in the development of diffusion models, exemplified by advancements such as Stable Diffusion, have underscored their remarkable prowess in generating visually compelling images. However, the imperative of achieving a seamless alignment between the generated image and the provided prompt persists as a formidable challenge. This paper traces the root of these difficulties to invalid initial noise, and proposes a solution in the form of Initial Noise Optimization (INITNO), a paradigm that refines this noise. Considering text prompts, not all random noises are effective in synthesizing semantically-faithful images. We design the cross-attention response score and the self-attention conflict score to evaluate the initial noise, bifurcating the initial latent space into valid and invalid sectors. A strategically crafted noise optimization pipeline is developed to guide the initial noise towards valid regions. Our method, validated through rigorous experimentation, shows a commendable proficiency in generating images in strict accordance with text prompts. Our code is available at https://***/xiefan-guo/initno. Copyright © 2024, The Authors. All rights reserved.

关键词： Diffusion

Why Does Little Robustness Help? A Further Step Towards Understanding Adversarial Transferability

学校读者我要写书评

暂无评论

Why Does Little Robustness Help? A Further Step Towards Unde...

IEEE Symposium on Security and Privacy

作者： Yechao Zhang Shengshan Hu Leo Yu Zhang Junyu Shi Minghui Li Xiaogeng Liu Wei Wan Hai Jin National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Hubei Engineering Research Center on Big Data Security Hubei Key Laboratory of Distributed System Security School of Cyber Science and Engineering Huazhong University of Science and Technology School of Information and Communication Technology Griffith University School of Software Engineering Huazhong University of Science and Technology Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology

ISBN: (数字)9798350331301

ISBN: (纸本)9798350331318

Adversarial examples for deep neural networks (DNNs) are transferable: examples that successfully fool one white-box surrogate model can also deceive other black-box models with different architectures. Although a bunch of empirical studies have provided guidance on generating highly transferable adversarial examples, many of these findings fail to be well explained and even lead to confusing or inconsistent advice for practical *** this paper, we take a further step towards understanding adversarial transferability, with a particular focus on surrogate aspects. Starting from the intriguing "little robustness" phenomenon, where models adversarially trained with mildly perturbed adversarial samples can serve as better surrogates for transfer attacks, we attribute it to a trade-off between two dominant factors: model smoothness and gradient similarity. Our research focuses on their joint effects on transferability, rather than demonstrating the separate relationships alone. Through a combination of theoretical and empirical analyses, we hypothesize that the data distribution shift induced by off-manifold samples in adversarial training is the reason that impairs gradient *** on these insights, we further explore the impacts of prevalent data augmentation and gradient regularization on transferability and analyze how the trade-off manifests in various training methods, thus building a comprehensive blueprint for the regulation mechanisms behind transferability. Finally, we provide a general route for constructing superior surrogates to boost transferability, which optimizes both model smoothness and gradient similarity simultaneously, e.g., the combination of input gradient regularization and sharpness-aware minimization (SAM), validated by extensive experiments. In summary, we call for attention to the united impacts of these two factors for launching effective transfer attacks, rather than optimizing one while ignoring the other, and emphasize the

关键词： Training Privacy Buildings Closed box Minimization Data augmentation Robustness

Initno: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization

学校读者我要写书评

暂无评论

Initno: Boosting Text-to-Image Diffusion Models via Initial ...

Conference on computer Vision and Pattern Recognition (CVPR)

作者： Xiefan Guo Jinlin Liu Miaomiao Cui Jiankai Li Hongyu Yang Di Huang State Key Laboratory of Software Development Environment Beihang University Beijing China School of Computer Science and Engineering Beihang University Beijing China Institute of Artificial Intelligence Beihang University Beijing China Shanghai Artificial Intelligence Laboratory Shanghai China

ISBN: (数字)9798350353006

ISBN: (纸本)9798350353013

Recent strides in the development of diffusion models, ex-emplified by advancements such as Stable Diffusion, have underscored their remarkable prowess in generating visu-ally compelling images. However, the imperative of achieving a seamless alignment between the generated image and the provided prompt persists as a formidable challenge. This paper traces the root of these difficulties to invalid initial noise, and proposes a solution in the form of Initial Noise Optimization (INITNO), a paradigm that refines this noise. Considering text prompts, not all random noises are effective in synthesizing semantically-faithful images. We design the cross-attention response score and the selfattention conflict score to evaluate the initial noise, bifurcating the initial latent space into valid and invalid sectors. A strategically crafted noise optimization pipeline is developed to guide the initial noise towards valid regions. Our method, validated through rigorous experimentation, shows a commendable proficiency in generating images in strict accordance with text prompts. Our code is available at https://***/xiefan-guo/initno.

关键词： computer vision Navigation Instruments Noise Pipelines Text to image Aerospace electronics

PEFN: A Patches Enhancement and Hierarchical Fusion Network for Robust Vehicle Re-Identification

学校读者我要写书评

暂无评论

IEEE Internet of Things Journal 2025年

作者： He, Wenying Wang, Feiyu Bai, Yude Xiong, Neal N. Xu, Guangquan Guo, Fei Hebei University of Technology School of Artificial Intelligence Tianjin300401 China Hebei University of Technology Hebei Province Key Laboratory of Big Data Calculation Tianjin300130 China Tiangong University School of Software Tianjin300387 China Northeastern State University Department of Mathematics and Computer Science TahlequahOK74464 United States Tianjin300350 China Central South University School of Computer Science and Engineering Changsha410083 China

Vehicle Re-Identification (Re-ID), which is a significant application in the Internet of Things, aims to accurately retrieve the remaining images of a given vehicle across different cameras views. The improvement in vehicle Re-ID performance largely stems from better addressing the issues of inter-class similarity and intra-class variance. Existing methods, relying solely on max or average pooling after using attention modules, fail to obtain significantly complete and pure global and local features, and neglect the false guidance that some unique individual information on images bring to re-identification. Moreover, models combining global and local features have shown good results in vehicle Re-ID, but these successes neglect the interaction between features across different convolutional layers, resulting in the loss of crucial details for vehicle Re-ID. To tackle these issues, we introduce a Patches Enhancement and hierarchical Fusion Network (PEFN) based on a multi-branch architecture, divided into a Global and Local Attention Supplement (GLAS) branch, and an Enhanced Hierarchical feature fusion (EnHi) branch. The GLAS branch, through the Identity-related Feature Remodeling (IDFR) module’s staged supplementation of spatial and channel features, has achieved the enhancement of both global and local features and effectively mitigated the negative impacts of individual information. The EnHi branch enhances the robustness of feature representation by interacting hierarchical features. Extensive experiments on two large-scale vehicle re-identification datasets demonstrate that our PEFN method outperforms state-of-the-art vehicle re-identification approaches. Specifically, without utilizing extra data and re-ranking, our model achieves 85.15% mAP on the VeRi776 dataset. © 2014 IEEE.

关键词： Reamers

NeRFTAP: Enhancing Transferability of Adversarial Patches on Face Recognition using Neural Radiance Fields

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Liu, Xiaoliang Shen, Furao Han, Feng Zhao, Jian Nie, Changhai State Key Laboratory for Novel Software Technology Nanjing University China Department of Computer Science and Technology Nanjing University China School of Artificial Intelligence Nanjing University China School of Electronic Science and Engineering Nanjing University China

Face recognition (FR) technology plays a crucial role in various applications, but its vulnerability to adversarial attacks poses significant security concerns. Existing research primarily focuses on transferability to different FR models, overlooking the direct transferability to victim’s face images, which is a practical threat in real-world scenarios. In this study, we propose a novel adversarial attack method that considers both the transferability to the FR model and the victim’s face image, called NeRFTAP. Leveraging NeRF-based 3D-GAN, we generate new view face images for the source and target subjects to enhance transferability of adversarial patches. We introduce a style consistency loss to ensure the visual similarity between the adversarial UV map and the target UV map under a 0-1 mask, enhancing the effectiveness and naturalness of the generated adversarial face images. Extensive experiments and evaluations on various FR models demonstrate the superiority of our approach over existing attack techniques. Our work provides valuable insights for enhancing the robustness of FR systems in practical adversarial settings. Copyright © 2023, The Authors. All rights reserved.

关键词： Face recognition

Securely Fine-tuning Pre-trained Encoders Against Adversarial Examples

学校读者我要写书评

暂无评论

Securely Fine-tuning Pre-trained Encoders Against Adversaria...

IEEE Symposium on Security and Privacy

作者： Ziqi Zhou Minghui Li Wei Liu Shengshan Hu Yechao Zhang Wei Wan Lulu Xue Leo Yu Zhang Dezhong Yao Hai Jin National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology School of Software Engineering Huazhong University of Science and Technology Hubei Engineering Research Center on Big Data Security Hubei Key Laboratory of Distributed System Security School of Cyber Science and Engineering Huazhong University of Science and Technology School of Information and Communication Technology Griffith University

ISBN: (数字)9798350331301

ISBN: (纸本)9798350331318

With the evolution of self-supervised learning, the pre-training paradigm has emerged as a predominant solution within the deep learning landscape. Model providers furnish pre-trained encoders designed to function as versatile feature extractors, enabling downstream users to harness the benefits of expansive models with minimal effort through fine-tuning. Nevertheless, recent works have exposed a vulnerability in pre-trained encoders, highlighting their susceptibility to downstream-agnostic adversarial examples (DAEs) meticulously crafted by attackers. The lingering question pertains to the feasibility of fortifying the robustness of downstream models against DAEs, particularly in scenarios where the pre-trained encoders are publicly accessible to the *** this paper, we initially delve into existing defensive mechanisms against adversarial examples within the pre-training paradigm. Our findings reveal that the failure of current defenses stems from the domain shift between pre-training data and downstream tasks, as well as the sensitivity of encoder parameters. In response to these challenges, we propose Genetic Evolution-Nurtured Adversarial Fine-tuning (Gen-AF), a two-stage adversarial fine-tuning approach aimed at enhancing the robustness of downstream models. Gen-AF employs a genetic-directed dual-track adversarial fine-tuning strategy in its first stage to effectively inherit the pre-trained encoder. This involves optimizing the pre-trained encoder and classifier separately while incorporating genetic regularization to preserve the model’s topology. In the second stage, Gen-AF assesses the robust sensitivity of each layer and creates a dictionary, based on which the top-k robust redundant layers are selected with the remaining layers held fixed. Upon this foundation, we conduct evolutionary adaptability fine-tuning to further enhance the model’s generalizability. Our extensive experiments, conducted across ten self-supervised training methods and six d

关键词： Training Adaptation models Sensitivity Accuracy Self-supervised learning Genetics Feature extraction

DeepCL: Deep Change Feature Learning on Remote Sensing Images in the Metric Space

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Guo, Haonan Du, Bo Wu, Chen Han, Chengxi Zhang, Liangpei The State Key Laboratory of Information Engineering in Surveying Mapping and Remote Sensing Wuhan University Wuhan China The National Engineering Research Center for Multimedia Software Institute of Artificial Intelligence School of Computer Science Hubei Key Laboratory of Multimedia and Network Communication Engineering Wuhan University Wuhan China

Change detection (CD) is an important yet challenging task in the Earth observation field for monitoring Earth surface dynamics. The advent of deep learning techniques has recently propelled automatic CD into a technological revolution. Nevertheless, deep learning-based CD methods are still plagued by two primary issues: 1) insufficient temporal relationship modeling and 2) pseudo-change misclassification. To address these issues, we complement the strong temporal modeling ability of metric learning with the prominent fitting ability of segmentation and propose a deep change feature learning (DeepCL) framework for robust and explainable CD. Firstly, we designed a hard sample-aware contrastive loss, which reweights the importance of hard and simple samples. This loss allows for explicit modeling of the temporal correlation between bi-temporal remote sensing images. Furthermore, the modeled temporal relations are utilized as knowledge prior to guide the segmentation process for detecting change regions. The DeepCL framework is thoroughly evaluated both theoretically and experimentally, demonstrating its superior feature discriminability, resilience against pseudo changes, and adaptability to a variety of CD algorithms. Extensive comparative experiments substantiate the quantitative and qualitative superiority of DeepCL over state-of-the-art CD approaches. The source code will be made available at https://***/HaonanGuo/DeepCL. Copyright © 2023, The Authors. All rights reserved.

关键词： Change detection

Robust Synthetic-to-Real Transfer for Stereo Matching

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Jiawei Li, Jiahe Huang, Lei Yu, Xiaohan Gu, Lin Zheng, Jin Bai, Xiao School of Computer Science and Engineering State Key Laboratory of Complex & Critical Software Environment Jiangxi Research Institute Beihang University China SKLCCSE Institute of Artificial Intelligence Beihang University China School of Computing Macquarie University Australia RIKEN AIP Japan The University of Tokyo Japan

With advancements in domain generalized stereo matching networks, models pre-trained on synthetic data demonstrate strong robustness to unseen domains. However, few studies have investigated the robustness after fine-tuning them in real-world scenarios, during which the domain generalization ability can be seriously degraded. In this paper, we explore fine-tuning stereo matching networks without compromising their robustness to unseen domains. Our motivation stems from comparing Ground Truth (GT) versus Pseudo label (PL) for fine-tuning: GT degrades, but PL preserves the domain generalization ability. Empirically, we find the difference between GT and PL implies valuable information that can regularize networks during fine-tuning. We also propose a framework to utilize this difference for fine-tuning, consisting of a frozen Teacher, an exponential moving average (EMA) Teacher, and a Student network. The core idea is to utilize the EMA Teacher to measure what the Student has learned and dynamically improve GT and PL for fine-tuning. We integrate our framework with state-of-the-art networks and evaluate its effectiveness on several real-world datasets. Extensive experiments show that our method effectively preserves the domain generalization ability during fine-tuning. Code is available at: https://***/jiaw-z/DKT-Stereo. Copyright © 2024, The Authors. All rights reserved.

关键词： Machine learning

RADAP: A Robust and Adaptive Defense Against Diverse Adversarial Patches on Face Recognition

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Liu, Xiaoliang Shen, Furao Zhao, Jian Nie, Changhai State Key Laboratory for Novel Software Technology Nanjing University China Department of Computer Science and Technology Nanjing University China School of Artificial Intelligence Nanjing University China School of Electronic Science and Engineering Nanjing University China

Face recognition (FR) systems powered by deep learning have become widely used in various applications. However, they are vulnerable to adversarial attacks, especially those based on local adversarial patches that can be physically applied to real-world objects. In this paper, we propose RADAP, a robust and adaptive defense mechanism against diverse adversarial patches in both closed-set and open-set FR systems. RADAP employs innovative techniques, such as FCutout and F-patch, which use Fourier space sampling masks to improve the occlusion robustness of the FR model and the performance of the patch segmenter. Moreover, we introduce an edge-aware binary cross-entropy (EBCE) loss function to enhance the accuracy of patch detection. We also present the split and fill (SAF) strategy, which is designed to counter the vulnerability of the patch segmenter to complete white-box adaptive attacks. We conduct comprehensive experiments to validate the effectiveness of RADAP, which shows significant improvements in defense performance against various adversarial patches, while maintaining clean accuracy higher than that of the undefended Vanilla model. Copyright © 2023, The Authors. All rights reserved.

关键词： Face recognition