TEXT-TO-IMAGE RECTIFIED FLOW AS PLUG-AND-PLAY PRIORS

作     者：Yang, Xiaofeng Chen, Cheng Yang, Xulei Liu, Fayao Lin, Guosheng 

作者机构：College of Computing and Data Science Nanyang Technological University Singapore Institute for Infocomm Research A*STAR Singapore 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2024年

核心收录：

主　　题：Diffusion 

摘      要：Large-scale diffusion models have achieved remarkable performance in generative tasks. Beyond their initial training applications, these models have proven their ability to function as versatile plug-and-play priors. For instance, 2D diffusion models can serve as loss functions to optimize 3D implicit models. Rectified Flow, a novel class of generative models, has demonstrated superior performance across various domains. Compared to diffusion-based methods, rectified flow approaches surpass them in terms of generation quality and efficiency. In this work, we present theoretical and experimental evidence demonstrating that rectified flow based methods offer similar functionalities to diffusion models — they can also serve as effective priors. Besides the generative capabilities of diffusion priors, motivated by the unique time-symmetry properties of rectified flow models, a variant of our method can additionally perform image inversion. Experimentally, our rectified flow based priors outperform their diffusion counterparts — the SDS and VSD losses — in text-to-3D generation. Our method also displays competitive performance in image inversion and editing. Code is available at: https://***/yangxiaofeng/rectified flow prior. © 2024, CC BY.