Ipdm: identity preserving diffusion model for face sketch and photo synthesis

作     者：Tang, Duoxun Jiang, Xinhang Zhang, Ying Dai, Yuhang Lin, Ye 

作者机构：Sichuan Agr Univ Coll Sci Yaan 625014 Sichuan Peoples R China Sichuan Agr Univ Coll Informat Engn Yaan 625014 Sichuan Peoples R China 

出 版 物：《MACHINE VISION AND APPLICATIONS》 (Mach Vision Appl)

年 卷 期：2025年第36卷第2期

页      面：1-14页

核心收录：

学科分类：0808[工学-电气工程] 08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：The 2024 Dual-Branch Free Exploration Project of Sichuan Agricultural University [2024ZYTS033] Sichuan Educational Information Technology and Scientific Research Project [DSJZXKT213] Research Start-up Funds of Sichuan Agricultural University [031-2222996009] 

主　　题：Face sketch synthesis Identity preservation Diffusion model Generative model Image processing 

摘      要：Face sketch and photo synthesis is widely applied in industry and information fields, such as entertainment business and heterogeneous face retrieval. The key challenge lies in completing a face transformation with both good visual effects and face identity preservation. However, existing methods are still difficult to obtain a good synthesis due to the large model gap between the two different face domains. Recently, diffusion models have achieved great success in image synthesis, which allows us to extend its application in such a face generation task. Thus, we propose IPDM, which constructs a mapping of latent representation for domain-adaptive face features. The other proposed IDP utilizes auxiliary features to correct the latent features through their directions and supplementary identity information, so that the generation can keep face identity unchanged. The various evaluation results show that our method is superior to state-of-the-art methods in both identity preservation and visual effects.