Exploring the effects of synthetic data generation: a case study on autonomous driving for semantic segmentation

作     者：Silva, Manuel Seoane, Antonio Mures, Omar A. Lopez, Antonio M. Iglesias-Guitian, Jose A. 

作者机构：CITIC Ctr ICT Res La Coruna Spain Univ A Coruna La Coruna Spain Univ Autonoma Barcelona Comp Vis Ctr Bellaterra Catalonia Spain Univ Autonoma Barcelona Comp Sci Dept Bellaterra Spain Univ Autonoma Barcelona Comp Sci Dept Escola Engn Edif Q Bellaterra 08193 Spain 

出 版 物：《VISUAL COMPUTER》 (Visual Comput)

年 卷 期：2025年

页      面：1-19页

核心收录：

学科分类：08[工学] 0835[工学-软件工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：CRUE-CSIC agreement with Springer Nature MCIN/AEI [PID2020-115734RB-C22, PID2020-115734RB- C21] Xunta de Galicia (Conselleria de Cultura, Educacion e Universidade) [ED481A-2023-191] ICREA under the ICREA Academia Program Generalitat de Catalunya CERCA Program UDC-Inditex InTalent program Ministry of Science and Innovation [AEI/RYC2018-025385-I] Xunta de Galicia [ED431F 2021/11] 

主　　题：Computer graphics Rendering Autonomous driving Semantic segmentation 

摘      要：Rendering 3D virtual scenarios has become a popular alternative for generating per-pixel-labeled image datasets, especially in fields like autonomous driving. The approach is valuable for training neural perception models, such as semantic segmentation models, particularly when data might be scarce, expensive, or difficult to collect. However, fundamental questions persist within the research community regarding the generation and processing of these synthetic images, particularly a better understanding of the key factors influencing the performance of deep learning models trained with such synthetic images. In response, we conducted a series of experiments to elucidate the impact that common aspects involved in the generation of rendered synthetic images may have on the performance of neural semantic segmentation tasks. Our study used a recent autonomous driving synthetic dataset as our main testbed, allowing us to investigate the effect of different approaches when modeling their geometric, material, and lighting details. We also studied the impact of rendering noise, typically produced by path-tracing algorithms, as well as the impact of using different color transformations and tonemapping algorithms.