Integrating LidAR and camera information has been a widely adopted approach for 3dobjectdetection in autonomous driving. Nevertheless, the unexplored potential of multi-modal fusion remains in the realm of offline 4...
详细信息
Integrating LidAR and camera information has been a widely adopted approach for 3dobjectdetection in autonomous driving. Nevertheless, the unexplored potential of multi-modal fusion remains in the realm of offline 4ddetection. We experimentally find that the root lies in two reasons: (1) the sparsity of point clouds poses a challenge in extracting long-term image features and thereby results in information loss. (2) some of the LidAR points may be obstructed in the image, leading to incorrect image features. To tackle these problems, we first propose a simple yet effective offline multi-modal 3dobjectdetection method, named Fusion4dAL, for 4d auto-labeling with long-term multi-modal sequences. Specifically, in order to address the sparsity of points within objects, we propose a multi-modal mixed feature fusion module (MMFF). In the MMFF module, we introduce virtual points based on a dense 3d grid and combine them with real LidAR points. The mixed points are then utilized to extract dense point-level image features, thereby enhancing multi-modal feature fusion without being constrained by the sparse real LidAR points. As to the obstructed LidAR points, we leverage the occlusion relationship among objects to ensure depth consistency between LidAR points and their corresponding depth feature maps, thus filtering out erroneous image features. In addition, we define a virtual point loss (VP Loss) to distinguish different types of mixed points and preserve the geometric shape of objects. Furthermore, in order to promote long-term receptive field and capture finer-grained features, we propose a global point attention decoder with a box-level self-attention module and a global point attention module. Finally, comprehensive experiments show that Fusion4dAL outperforms state-of-the-art offline3ddetection methods on nuScenes and Waymo dataset.
The 4d auto-labeling system, with its potential to enhance data annotation efficiency for 3dobjectdetection, has garnered significant attention. However, its adoption has been hampered by the high costs associated w...
详细信息
暂无评论