On the Robustness of Latent Diffusion Models

作     者：Zhang, Jianping Xu, Zhuoer Cui, Shiwen Meng, Changhua Wu, Weibin Lyu, Michael R. 

作者机构：Department of Computer Science and Engineering The Chinese University of Hong Kong Hong Kong Tiansuan Lab Antgroup China School of Software Engineering Sun Yat-sen University China 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2023年

核心收录：

主　　题：Machine learning 

摘      要：Latent diffusion models achieve state-of-the-art performance on a variety of generative tasks, such as image synthesis and image editing. However, the robustness of latent diffusion models is not well studied. Previous works only focus on the adversarial attacks against the encoder or the output image under white-box settings, regardless of the denoising process. Therefore, in this paper, we aim to analyze the robustness of latent diffusion models more thoroughly. We first study the influence of the components inside latent diffusion models on their white-box robustness. In addition to white-box scenarios, we evaluate the black-box robustness of latent diffusion models via transfer attacks, where we consider both prompt-transfer and model-transfer settings and possible defense mechanisms. However, all these explorations need a comprehensive benchmark dataset, which is missing in the literature. Therefore, to facilitate the research of the robustness of latent diffusion models, we propose two automatic dataset construction pipelines for two kinds of image editing models and release the whole dataset. Our code and dataset are available at https://***/jpzhang1810/LDM-Robustness. Copyright © 2023, The Authors. All rights reserved.