Light Field Image Compression Using Generative Adversarial Network-Based View Synthesis

作     者：Jia, Chuanmin Zhang, Xinfeng Wang, Shanshe Wang, Shiqi Ma, Siwei 

作者机构：Peking Univ Inst Digital Media Beijing 100871 Peoples R China Univ Southern Calif Ming Hsieh Dept Elect Engn Los Angeles CA 90089 USA City Univ Hong Kong Dept Comp Sci Hong Kong Peoples R China 

出 版 物：《IEEE JOURNAL ON EMERGING AND SELECTED TOPICS IN CIRCUITS AND SYSTEMS》 (IEEE J. Emerg. Sel. Top. Circuits Syst.)

年 卷 期：2019年第9卷第1期

页      面：177-189页

核心收录：

学科分类：0808[工学-电气工程] 08[工学] 

基　　金：National Natural Science Foundation of China Top-Notch Young Talents Program of China Hong Kong RGC Early Career Scheme [9048122 (CityU 21211018)] City University of Hong Kong [7200539/CS] China Scholarship Council high-performance computing platform of Peking University 

主　　题：Light field image compression SAI synthesis adversarial learning hierarchical coding 

摘      要：Light field (LF) has become an attractive representation of immersive multimedia content for simultaneously capturing both the spatial and angular information of the light rays. In this paper, we present a LF image compression framework driven by a generative adversarial network (GAN)-based sub-aperture image (SAI) generation and a cascaded hierarchical coding structure. Specifically, we sparsely sample the SAIs in LF and propose the GAN of LF (LF-GAN) to generate the unsampled SAIs by analogy with adversarial learning conditioned on its surrounding contexts. In particular, the LF-GAN learns to interpret both the angular and spatial context of the LF structure and, meanwhile, generates intermediate hypothesis for the unsampled SAIs in a certain position. Subsequently, the sampled SAIs and the residues of the generated-unsampled SAIs are re-organized as pseudo-sequences and compressed by standard video codecs. Finally, the hierarchical coding structure is adopted for the sampled SAI to effectively remove the inter-view redundancies. During the training process of IF-GAN, the pixel-wise Euclidean loss and the adversarial lass are chosen as the optimization objective, such that sharp textures with less blurring in details can be produced. Extensive experimental results show that the proposed LF-GAN-based LF image compression framework outperforms the state-of-the-art learning-based IF image compression approach with on average 4.9% BD-rate reductions over multiple LF datasets.