A Multitask Network for Multiview Stereo Reconstruction: When Semantic Consistency-Based Clustering Meets Depth Estimation Optimization

作     者：Huang, Xin Zhang, Shulei Li, Jiayi Wang, Leiguang 

作者机构：Wuhan Univ Sch Remote Sensing & Informat Engn Wuhan 430079 Peoples R China Hubei Luojia Lab Wuhan 430079 Peoples R China Southwest Forestry Univ Inst Big Data & Artificial Intelligence Kunming 650024 Peoples R China Southwest Forestry Univ Key Lab State Forestry & Grassland Adm Forestry & Kunming 650024 Peoples R China 

出 版 物：《IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING》 (IEEE Trans Geosci Remote Sens)

年 卷 期：2024年第62卷

页      面：1页

核心收录：

学科分类：0808[工学-电气工程] 1002[医学-临床医学] 08[工学] 0708[理学-地球物理学] 0816[工学-测绘科学与技术] 

基　　金：Major Scientific and Technological Projects of Yunnan Province 

主　　题：Clustering algorithm deep learning depth map-based multiview stereo (MVS) reconstruction remote sensing imagery 

摘      要：We propose a novel network for multiview stereo (MVS) reconstruction in the field of remote sensing, which considers clustering-based semantic consistency into depth estimation optimization, referred to as CSC-MVS. In this approach, high-level semantic information acquired from multiple views is used to construct semantic consistency and assist in guiding the optimization of the MVS network. Specifically, the nonnegative matrix factorization (NMF) branch and the deep spectral decomposition (DSD) branch are designed to generate local and global semantic guidance, respectively. We then propose an uncertainty multitask optimization method to adaptively combine matching and semantic metrics. The performance of CSC-MVS is evaluated on representative benchmarks, including the WHU TLC dataset and LuoJia-MVS dataset, demonstrating its effectiveness and generality across diverse remote sensing scenarios. Comprehensive experimental results show that our CSC-MVS significantly improves the performance of various MVS baseline networks and achieves notable accuracy in depth reconstruction. We also conduct ablation studies to validate the rationality of each component and sensitivity analysis to confirm the robustness and adaptability of our proposed method. The code is available at https://***/zsl-whu/csc-mvs.