SP-Det: Leveraging Saliency Prediction for Voxel-Based 3D Object Detection in Sparse Point Cloud

作     者：An, Pei Duan, Yucong Huang, Yuliang Ma, Jie Chen, Yanfei Wang, Liheng Yang, You Liu, Qiong 

作者机构：Wuhan Inst Technol Sch Elect & Informat Engn Wuhan 430205 Peoples R China Huazhong Univ Sci & Technol Sch Artificial Intelligence & Automat Wuhan 430074 Peoples R China Huazhong Univ Sci & Technol Sch Elect Informat & Commun Wuhan 430074 Peoples R China Wuhan Natl Lab Optoelect Wuhan 430074 Peoples R China 

出 版 物：《IEEE TRANSACTIONS ON MULTIMEDIA》 (IEEE Trans Multimedia)

年 卷 期：2024年第26卷

页      面：2795-2808页

核心收录：

学科分类：0810[工学-信息与通信工程] 0808[工学-电气工程] 08[工学] 0835[工学-软件工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：National Key Ramp D Program of China 

主　　题：Three-dimensional displays Task analysis Point cloud compression Feature extraction Laser radar Detectors Object detection 3D object detection 3D point cloud autonomous driving light detection and ranging saliency prediction voxel 

摘      要：Voxel is one of the common structural representation of 3D point cloud. Due to the sparsity of point cloud generated by light detection and ranging (LiDAR), there is the extreme imbalance in the foreground and background voxels. It decreases the accuracy of 3D object detection, has the negative effect on intelligent driving safety. To overcome this problem, we present a saliency prediction based 3D object detector SP-Det in this article. Although foreground voxels have the sufficient feature of object, it is difficult to localize the foreground region from voxel space with the larger background region. We design an auxiliary learning task, saliency prediction (SP). It benefits 3D detector in identifying the foreground region. SP task uses label diffusion to alleviate the label imbalance. It reduces the learning difficulty of saliency in voxel and bird s eye view (BEV) spaces. After that, to strengthen feature interaction from the sparse foreground region, we design saliency fusion (SF) module to fuse the learning result in SP task. It utilizes voxel and BEV saliency maps as progressive attention to resist the redundant feature from background region. To aggregate more foreground feature inside 3D and BEV region of interest (RoI), we design hybrid grid maps based RoI pooling (Hybrid-RoI pooling). Experiments are conducted in STF dataset. The adverse weather enlarges the sparsity of LiDAR point cloud, increasing the difficulty of object detection. SP-Det identifies and leverages foreground region, and achieves the performance better than the current methods. Hence, we believe that SP-Det benefits to LiDAR based 3D scene understanding in the adverse weather.