StyleGAN-∞: Extending StyleGAN to Arbitrary-Ratio Translation with StyleBook

作     者：Dai, Yihua Xiang, Tianyi Deng, Bailin Du, Yong Cai, Hongmin Qin, Jing He, Shengfeng 

作者机构：South China University Of Technology School Of Computer Science And Engineering Guangzhou China Singapore Management University School Of Computing And Information Systems Singapore Cardiff University School Of Computer Science And Informatics United Kingdom Ocean University Of China Faculty Of Information Science And Engineering Qingdao China Hong Kong Polytechnic University School Of Nursing Hong Kong Hong Kong 

出 版 物：《IEEE Transactions on Visualization and Computer Graphics》 (IEEE Trans Visual Comput Graphics)

年 卷 期：2024年

核心收录：

学科分类：0808[工学-电气工程] 08[工学] 0835[工学-软件工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

主　　题：Generative adversarial networks 

摘      要：Although pre-trained large-scale generative models StyleGAN series have proven to be effective in various editing and translation tasks, they are limited to pre-defined fixed aspect ratio. To overcome this limitation, we propose StyleGAN-∞, a model that enables pre-trained StyleGAN to perform arbitrary-ratio conditional synthesis. Our key insight is to distill the expressive StyleGAN features into a StyleBook, such that an arbitrary-ratio condition can be translated to other forms by properly assembling pre-defined StyleBook vectors. To learn and leverage the StyleBook, we employ a network with three distinct stages, each corresponding to StyleBook extraction, StyleBook correspondence learning, and arbitrary-ratio synthesis. Extensive experiments on various conditional synthesis tasks, like super-resolution, sketch synthesis, and semantic synthesis, demonstrate superior performances over state-of-the-art image-to-image translation methods. Moreover, our model can easily generate megapixel images in diverse modalities by taking advantage of different pre-trained StyleGAN models. © 1995-2012 IEEE.