Surface-Continuous Scene Representation for Light Field Depth Estimation via Planarity Prior

作     者：Chen, Rongshan Sheng, Hao Yang, Da Cui, Zhenglong Cong, Ruixuan 

作者机构：Beihang Univ Sch Comp Sci & Engn State Key Lab Virtual Real Technol & Syst Beijing 100191 Peoples R China Beihang Univ Hangzhou Int Innovat Inst Key Lab Data Sci & Intelligent Comp Hangzhou 311115 Zhejiang Peoples R China Macao Polytech Univ Fac Appl Sci Macau Peoples R China 

出 版 物：《IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY》 (IEEE Trans Circuits Syst Video Technol)

年 卷 期：2025年第35卷第5期

页      面：5051-5066页

核心收录：

学科分类：0808[工学-电气工程] 08[工学] 

基　　金：National Key Research and Development Program of China [2022YFC3803600] National Natural Science Foundation of China Open Fund of the State Key Laboratory of Software Development Environment [SKLSDE2023ZX-11] Research Start-Up Funds of Hangzhou International Innovation Institute of Beihang University [2024KQ012] Haiyou Plan Fund 

主　　题：Depth measurement Surface reconstruction Cameras Light fields Three-dimensional displays Image reconstruction Estimation Surface treatment Deep learning Costs Light field depth estimation surface-continuous plane regular sampling operator PlaneNet 

摘      要：Light field (LF) imaging captures both spatial and angular information of the real world, enabling precise depth estimation. However, images are merely discrete expressions of scenes. Limited by imaging technology, LF camera cannot capture the infinite rays emitted by scenes, leading to the discrete information storage (e.g. pixel). Consequently, previous deep learning methods have encountered challenges in accurately extracting depth information from LF images. In this paper, we investigate a surface-continuous scene representation using planarity prior and design PlaneNet, a Plane-based Network that successfully generates highly detailed depth maps for real scenes. Specifically, inspired by the plane assumption that real-world scenes generally yield piecewise smooth surfaces, we refine it to the pixel level for continuous surface approximation, which can overcome the limitations of discrete representation. Rather than explicitly parameterizing planes as multiple coefficients, we propose a novel plane regular sampling operator (PRSO), enabling the network to fit smooth depth surfaces easily. To explore the role of our theory at the feature level, we also introduce PRSO into the intermediate layers of PlaneNet. Experiments show that our method achieves state-of-the-art performance on both synthetic and real-world LF scenes, ranking 1st (MSE) on the HCI 4D Light Field benchmark. Furthermore, we explore the utilization of our representation in multiple LF depth estimation networks, and experiments demonstrate improved performance when surface-continuous representation is applied. Code is available at https://***/crs904620522/PlaneNet.