Road segmentation with image-LiDAR data fusion in deep neural network

作     者：Liu, Huafeng Yao, Yazhou Sun, Zeren Li, Xiangrui Jia, Ke Tang, Zhenming 

作者机构：Nanjing Univ Sci & Technol Sch Comp Sci & Engn Nanjing 210094 Peoples R China Chengdu Univ Informat Technol Sch Comp Sci Chengdu Peoples R China 

出 版 物：《MULTIMEDIA TOOLS AND APPLICATIONS》 (多媒体工具和应用)

年 卷 期：2020年第79卷第47-48期

页      面：35503-35518页

核心收录：

学科分类：0808[工学-电气工程] 08[工学] 0835[工学-软件工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：National Natural Science Foundation of China, NSFC NNSF NNSFC, (61473154) National Natural Science Foundation of China, NSFC NNSF NNSFC 

主　　题：Road segmentation Data fusion Deep learning 

摘      要：Robust road segmentation is a key challenge in self-driving research. Though many image based methods have been studied and high performances in dataset evaluations have been reported, developing robust and reliable road segmentation is still a major challenge. Data fusion across different sensors to improve the performance of road segmentation is widely considered an important and irreplaceable solution. In this paper, we propose a novel structure to fuse image and LiDAR point cloud in an end-to-end semantic segmentation network, in which the fusion is performed at decoder stage instead of at, more commonly, encoder stage. During fusion, we improve the multi-scale LiDAR map generation to increase the precision of multi-scale LiDAR map by introducing pyramid projection method. Additionally, we adapted the multi-path refinement network with our fusion strategy and improve the road prediction compared with transpose convolution with skip layers. Our approach has been tested on KITTI ROAD dataset and have a competitive performance.