End-to-End Three-Dimensional Reconstruction of Transparent Objects with Multiple Optimization Strategies Under Limited Constraints

作     者：Sha, Xiaopeng Si, Xiaopeng Li, Wenchao Zhu, Yujie Wang, Shuyu Zhao, Yuliang 

作者机构：School of Control Engineering Northeastern University at Qinhuangdao Qinhuangdao066004 China Hebei Key Laboratory of Micro-Nano Precision Optical Sensing and Measurement Technology Qinhuangdao066004 China 

出 版 物：《SSRN》 

年 卷 期：2023年

核心收录：

主　　题：Image segmentation 

摘      要：Reconstructing transparent objects with limited constraints has long been considered a highly challenging problem. Due to the complex interaction between transparent objects and light, which involves intricate refraction and reflection relationships, traditional three-dimensional (3D) reconstruction methods are less than effective for transparent objects. To address this issue, this study proposes a 3D reconstruction method specifically designed for transparent objects. Incorporating multiple optimization strategies, the method works under limited constraints to achieve the end-to-end reconstruction of transparent objects with only a few transparent object images in any known environment, without the need for specific data collection devices or environments. The proposed method makes use of automatic image segmentation and modifies the network interface and structure of the PointNeXt algorithm to introduce the TransNeXt network, which enhances normal features, optimizes weight attenuation, and employs a preheating cosine annealing learning rate. We use several steps to reconstruct the complete 3D shape of transparent objects. First, we initialize the transparent shape with a visual hull reconstructed with the contours obtained by the TOM-Net. Then, we construct the normal reconstruction network to estimate the normal values. Finally, we reconstruct the complete 3D shape using the TransNeXt network. Multiple experiments show that the TransNeXt network exhibits superior reconstruction performance to other networks and can effectively perform the end-to-end reconstruction of transparent objects even under limited constraints. © 2023, The Authors. All rights reserved.