检索结果-内蒙古大学图书馆

Fully Sparse Fusion for 3d object detection

IEEE TRANSACTIONS ON PATTERN ANALYSIS ANd MACHINE INTELLIGENCE 2024年第11期46卷 7217-7231页

作者： Li, Yingyan Fan, Lue Liu, Yang Huang, Zehao Chen, Yuntao Wang, Naiyan Zhang, Zhaoxiang Chinese Acad Sci CASIA Inst Automat Ctr Researchon Intelligent Percept & Comp CRIPAC State Key Lab Multimodal Artificial Intelligence S Beijing 100190 Peoples R China Univ Chinese Acad Sci UCAS Sch Future Technol Beijing 100049 Peoples R China TuSimple Beijing 100020 Peoples R China Chinese Acad Sci HKISICAS Hong Kong Inst Sci & Innovat Ctr Artificial Intelligence & Robot Hong Kong Peoples R China

Currently prevalent multi-modal 3d detection methods rely on dense detectors that usually use dense Bird's-Eye-View (BEV) feature maps. However, the cost of such BEV feature maps is quadratic to the detection range, making it not scalable for long-range detection. Recently, LidAR-only fully sparse architecture has been gaining attention for its high efficiency in long-range perception. In this paper, we study how to develop a multi-modal fully sparse detector. Specifically, our proposed detector integrates the well-studied 2d instance segmentation into the LidAR side, which is parallel to the 3d instance segmentation part in the LidAR-only baseline. The proposed instance-based fusion framework maintains full sparsity while overcoming the constraints associated with the LidAR-only fully sparse detector. Our framework showcases state-of-the-art performance on the widely used nuScenes dataset, Waymo Open dataset, and the long-range Argoverse 2 dataset. Notably, the inference speed of our proposed method under the long-range perception setting is 2.7x faster than that of other state-of-the-art multimodal 3d detection methods.

关键词： Three-dimensional displays Feature extraction Laser radar Cameras detectors Instance segmentation Point cloud compression 3d object detection multi-sensor fusion fully sparse architecture autonomous driving long-range perception

来源：评论

学校读者我要写书评

暂无评论

Automatic Pseudo-LidAR Annotation: Generation of Training data for 3d object detection Networks

引用

IEEE ACCESS 2024年 12卷 14227-14237页

作者： Oh, Changsuk Jang, Youngseok Shim, dongseok Kim, Changhyeon Kim, Junha Kim, H. Jin Seoul Natl Univ Dept Aerosp Engn Seoul 08826 South Korea Seoul Natl Univ Artificial Intelligence Inst Dept Mech & Aerosp Engn Seoul 08826 South Korea Seoul Natl Univ Interdisciplinary Program Artificial Intelligence Seoul 08826 South Korea Samsung Res Seoul 06765 South Korea

object detection in 3d is a key ingredient of various autonomous systems. Many 3d object detection methods rely on LidAR, as it is robust to illumination conditions and provides accurate distance measurements. To apply LidAR-based 3d object detection networks for new objects, we need new training datasets. However, because labeling target objects with 3d bounding boxes in LidAR point clouds requires significant resources and open datasets contain annotations of only car-related classes, it is challenging to deploy LidAR-based 3d object detectors for detecting objects not related to cars. We propose a system that automatically generates annotated pseudo-LidAR (APL) data, which requires only stereo images to synthesize 3d bounding box annotations and pseudo-LidAR points. Using the proposed method, we can dramatically reduce efforts and time for generating a LidAR-based 3d object detection dataset. By utilizing classes in 2d image datasets, the proposed framework can annotate diverse objects beyond limited classes of existing LidAR-based 3d object detection datasets. To verify the capability of the synthesized training data, we train 3d object detection networks with the APL data of new classes. The experiments show that the 3d object detection networks trained on the APL data can detect objects of the new classes in LidAR point clouds, which demonstrates that the proposed method can help LidAR-based 3d object detectors operate for various objects not covered in existing LidAR-based 3d object detection datasets.

关键词： 3d object detection light detection and ranging (LidAR) pseudo-LidAR point cloud

来源：评论

学校读者我要写书评

暂无评论

VPC-VoxelNet: multi-modal fusion 3d object detection networks based on virtual point clouds

引用

INTERNATIONAL JOURNAL OF MULTIMEdIA INFORMATION RETRIEVAL 2025年第1期14卷 1-11页

作者： Zhang, Qiang Shi, Qin Cheng, Teng Zhang, Junning Chen, Jiong Hefei Univ Technol Engn Res Ctr Intelligent Transportat & Cooperat Ve Key Lab Automated Vehicle Safety Technol Anhui Pro Hefei 230009 Peoples R China Chery Automobile Co Ltd Wuhu 241000 Peoples R China Natl Univ Def Technol Sch Elect Countermeasures Hefei 230041 Peoples R China Nio Automot Technol Anhui Co LTD Hefei 230041 Peoples R China

To address the impact of sparsity and disorder of point clouds on object detection accuracy, this paper proposes a multi-modal fusion network VPC-VoxelNet based on virtual point clouds. Firstly, virtual point clouds are constructed using image detection object information to increase the density of point clouds, thus improving the performance of object features;Secondly, increasing the dimensionality of point cloud features, distinguishing virtual point clouds and avoiding the accumulation of multi model errors;Finally, an optimized loss function such as the scale factor of the virtual point cloud is used to improve the training efficiency of the multi-modal network. The object detection network, VPC-VoxelNet, was tested on the KITTI dataset, and the detection accuracy was better than that of the classical 3d point cloud detection network and certain multi-modal information fusion networks, with a vehicle detection accuracy of 86.9%.

关键词： 3d object detection Multi-modal fusion Virtual point cloud Adaptive optimizer

来源：评论

学校读者我要写书评

暂无评论

MSFNet3d: Monocular 3d object detection via dual-Branch depth-Consistent Fusion and Semantic-Guided Point Cloud Refinement

引用

WORLd ELECTRIC VEHICLE JOURNAL 2025年第3期16卷 173-173页

作者： Yang, Rong You, Zhijie Luo, Renhui Guangxi Univ Coll Mech Engn Nanning 530004 Peoples R China

The rapid development of autonomous driving has underscored the pivotal role of 3d perception. Monocular 3d object detection, as a cost-effective alternative to expensive lidar systems, is garnering increasing attention. However, existing pseudo-lidar methods encounter challenges such as coarse quality and insufficient semantic information when generating 3d point clouds from monocular images. To address these issues, this paper introduces MSFNet3d, which aims to overcome the quality limitations of pseudo-lidar point cloud. Our contributions are threefold: (1) We introduce a dual-branch network to optimize depth maps and propose a multi-scale channel spatial attention module (MS_CBAM). This module captures multi-scale geometric features through a hierarchical feature pyramid and an adaptive weight allocation mechanism, thereby addressing the scale sensitivity inherent in traditional attention mechanisms. (2) We propose a consistency-weighted fusion strategy that employs local gradient consistency analysis and differentiable weighted optimization to achieve a pixel-level fusion of image and depth features. This approach reduces feature conflicts within the dual-branch network and enhances the model's robustness in complex scenes. (3) We introduce a semantic-guided pseudo-point cloud enhancement method that leverages an instance segmentation network to extract object-specific semantic regions and generate high-confidence point cloud, consequently improving the accuracy of object detection. Experiments on the KITTI dataset show that the proposed method performs excellently under various detection challenges, achieving an average precision of 18.87% in the 3d detection of car objects, which is a 1.67% improvement over the original model. The method also shows good performance in detecting pedestrians and cyclists. The proposed framework can provide economical and reliable 3d perception for mass-produced electric vehicles.

关键词： autonomous driving 3d object detection pseudo-point cloud attention mechanism

来源：评论

学校读者我要写书评

暂无评论

ImFusion: Boosting Two-Stage 3d object detection via Image Candidates

引用

IEEE SIGNAL PROCESSING LETTERS 2024年 31卷 241-245页

作者： Tao, Manli Zhao, Chaoyang Wang, Jinqiao Tang, Ming Chinese Acad Sci Inst Automat Beijing 100190 Peoples R China Univ Chinese Acad Sci Beijing 100049 Peoples R China ObjectEye Inc Beijing 100000 Peoples R China

Multi-modal fusion methods combine the advantages of both point clouds and RGB images to boost the performance of 3d object detection. despite the significant progress, we find that existing two-stage multi-modal fusion methods suffer from the 3d proposal missing in the first stage and projected-style feature fusion mechanism. To solve these problems, we propose a two-stage multi-modal feature fusion network, which improves the recall rate of hard targets in the first stage of network with pseudo 3d proposals generated from image candidates. Then, considering the complementary information between similar image foreground features across multiple objects, we design a multi-modal cross-target fusion module to pay more attention to the foreground objects. It enables a 3d proposal can aggregate the semantic features of multiple image candidates belonging to the same category. Finally, these enhanced fused proposals are processed in the second stage to further boost the performance of 3d detector. Experimental results on SUN RGB-d and KITTI datasets show the effectiveness of our proposed method.

关键词： Three-dimensional displays Proposals object detection Feature extraction Point cloud compression Aggregates Sun 3d object detection image candidates pseudo 3d proposal target missing

来源：评论

学校读者我要写书评

暂无评论

Adaptive learning point cloud and image diversity feature fusion network for 3d object detection

引用

COMPLEX & INTELLIGENT SYSTEMS 2024年第2期10卷 2825-2837页

作者： Yan, Weiqing Liu, Shile Liu, Hao Yue, Guanghui Wang, Xuan Song, Yongchao Xu, Jindong Yantai Univ Sch Comp & Control Engn Yantai Peoples R China Shenzhen Univ Hlth Sci Ctr Sch Biomed Engn Shenzhen 518060 Peoples R China

3d object detection is a critical task in the fields of virtual reality and autonomous driving. Given that each sensor has its own strengths and limitations, multi-sensor-based 3d object detection has gained popularity. However, most existing methods extract high-level image semantic features and fuse them with point cloud features, focusing solely on consistent information from both sensors while ignoring their complementary information. In this paper, we present a novel two-stage multi-sensor deep neural network, called the adaptive learning point cloud and image diversity feature fusion network (APIdFF-Net), for 3d object detection. Our approach employs the fine-grained image information to complement the point cloud information by combining low-level image features with high-level point cloud features. Specifically, we design a shallow image feature extraction module to learn fine-grained information from images, instead of relying on deep layer features with coarse-grained information. Furthermore, we design a diversity feature fusion (dFF) module that transforms low-level image features into point-wise image features and explores their complementary features through an attention mechanism, ensuring an effective combination of fine-grained image features and point cloud features. Experiments on the KITTI benchmark show that the proposed method outperforms state-of-the-art methods.

关键词： 3d object detection LidAR point cloud Fine-grained image diversity feature fusion

来源：评论

学校读者我要写书评

暂无评论

A Unified Framework for Adversarial Patch Attacks Against Visual 3d object detection in Autonomous driving

引用

IEEE TRANSACTIONS ON CIRCUITS ANd SYSTEMS FOR VIdEO TECHNOLOGY 2025年第5期35卷 4949-4962页

作者： Wang, Jian Li, Fan He, Lijun Xi An Jiao Tong Univ Sch Informat & Commun Engn Shaanxi Key Lab Deep Space Explorat Intelligent In Xian 710049 Peoples R China

The rapid development of vision-based 3d perceptions, in conjunction with the inherent vulnerability of deep neural networks to adversarial examples, motivates us to investigate realistic adversarial attacks for the 3d detection models in autonomous driving scenarios. due to the perspective transformation from 3d space to the image and object occlusion, current 2d image attacks are difficult to generalize to 3d detectors and are limited by physical feasibility. In this work, we propose a unified framework to generate physically printable adversarial patches with different attack goals: 1) instance-level hiding-pasting the learned patches to any target vehicle allows it to evade the detection process;2) scene-level creating-placing the adversarial patch in the scene induces the detector to perceive plenty of fake objects. Both crafted patches are universal, which can take effect across a wide range of objects and scenes. To achieve above attacks, we first introduce the differentiable image-3d rendering algorithm that makes it possible to learn a patch located in 3d space. Then, two novel designs are devised to promote effective learning of patch content: 1) a Sparse object Sampling Strategy is proposed to ensure that the rendered patches follow the perspective criterion and avoid being occluded during training, and 2) a Patch-Oriented Adversarial Optimization is used to facilitate the learning process focused on the patch areas. Both digital and physical-world experiments are conducted and demonstrate the effectiveness of our approaches, revealing potential threats when confronted with malicious attacks. We also investigate the defense strategy using adversarial augmentation to further improve the model's robustness.

关键词： Three-dimensional printing detectors Visualization Rendering (computer graphics) Solid modeling Optimization Cameras Autonomous vehicles Perturbation methods Training Adversarial attack universal patch 3d object detection defense

来源：评论

学校读者我要写书评

暂无评论

SEGANet: 3d object detection with shape-enhancement and geometry-aware network

引用

COMPUTERS & ELECTRICAL ENGINEERING 2023年第1期110卷

作者： Zhou, Jing Hu, Yiyu Lai, Zhongyuan Wang, Tianjiang Jianghan Univ Sch Artificial Intelligence Wuhan 430056 Hubei Peoples R China Huazhong Univ Sci & Technol Sch Comp Sci & Technol Wuhan 430074 Hubei Peoples R China

3d object detection approaches from point clouds develop rapidly. However, the distribution of point clouds is unbalanced in the real scene, and thus the distant or occluded objects suffer from too few points to be perceived. This case damages the overall detection accuracy. Hence, we propose a novel two-stage 3d object detection framework, the Shape-Enhancement and Geometry-Aware Network (SEGANet), which aims to mitigate the negative impact of unbalanced point distribution for boosting detection performance. In stage 1, we first capture fine-grained structural knowledge with the assistance of point-wise features from voxels to generate proposals. And in stage 2, we construct a shape enhancement module to reconstruct complete surface points for objects within proposals, then establish an elaborate geometric relevance-aware Transformer module to aggregate high-correlated feature pairs of reconstructed-known parts and decode vital geometric relations of aggregated features. Thus, critical geometric clues are supplied for objects from the data and feature levels, achieving enhanced features for box refinement. Extensive experiments on KITTI and Waymo datasets show that SEGANet achieves low model complexity and excellent detection accuracy, especially leading the baseline method by 2.18% gain in overall detection accuracy and 1.8% gain in average accuracy of weakly sensing objects. This verifies that SEGANet effectively alleviates the impact of point imbalance to significantly boost detection performance.

关键词： 3d object detection Weakly sensing objects Point cloud completion Sparse attention Transformer

来源：评论

学校读者我要写书评

暂无评论

VPFNet: Improving 3d object detection With Virtual Point Based LidAR and Stereo data Fusion

引用

IEEE TRANSACTIONS ON MULTIMEdIA 2023年 25卷 5291-5304页

作者： Zhu, Hanqi deng, Jiajun Zhang, Yu Ji, Jianmin Mao, Qiuyu Li, Houqiang Zhang, Yanyong Univ Sci & Technol China Sch Comp Sci & Technol Hefei 230027 Peoples R China Univ Sci & Technol China Dept Elect Engn & Informat Sci Hefei 230027 Peoples R China

It has been well recognized that fusing the complementary information from depth-aware LidAR point clouds and semantic-rich stereo images would benefit 3d object detection. Nevertheless, it is non-trivial to explore the inherently unnatural interaction between sparse 3d points and dense 2d pixels. To ease this difficulty, the recent approaches generally project the 3d points onto the 2d image plane to sample the image data and then aggregate the data at the points. However, these approaches often suffer from the mismatch between the resolution of point clouds and RGB images, leading to sub-optimal performance. Specifically, taking the sparse points as the multi-modal data aggregation locations causes severe information loss for high-resolution images, which in turn undermines the effectiveness of multi-sensor fusion. In this paper, we present VPFNet -a new architecture that cleverly aligns and aggregates the point cloud and image data at the "virtual" points. Particularly, with their density lying between that of the 3d points and 2d pixels, the virtual points can nicely bridge the resolution gap between the two sensors, and thus preserve more information for processing. Moreover, we also investigate the data augmentation techniques that can be applied to both point clouds and RGB images, as the data augmentation has made non-negligible contribution towards 3d object detectors to date. We have conducted extensive experiments on KITTI dataset, and have observed good performance compared to the state-of-the-art methods. Remarkably, our VPFNet achieves 83.21% moderate $AP_{3d}$ and 91.86% moderate $AP_{BEV}$ on the KITTI test set. The network design also takes computation efficiency into consideration - we can achieve a FPS of 15 on a single NVIdIA RTX 2080Ti GPU.

关键词： 3d object detection multiple sensors point clouds stereo images

来源：评论

学校读者我要写书评

暂无评论

Multi-Source Features Fusion Single Stage 3d object detection With Transformer

引用

IEEE ROBOTICS ANd AUTOMATION LETTERS 2023年第4期8卷 2062-2069页

作者： Tong, Guofeng Li, Zheng Peng, Hao Wang, Yaqi Northeastern Univ Coll Informat Sci & Engn Shenyang 110819 Peoples R China

due to the high efficiency in extracting context information, voxel-based method is widely used in 3d object detection from point cloud. However, the quantization loss of geometric information is inevitable in the process of voxelization for raw point cloud, which may have a certain impact on final detection performance. To alleviate this problem, we propose a novel single-stage 3d detection algorithm named MFT-SSd for accurate 3d bounding box prediction. differ from most single-stage methods, our proposed framework combines point-based and voxel-based backbones, and extracts point, multi-scale voxel and BEV features as multi-source features, respectively. In order to enhance the correlation among different representation features, we propose a transformer feature fusion module with self-attention mechanism to fully integrate these multi-source features into richer point-wise features. Then, these fused point-wise features are sent to a candidate generation layer to generate a series of candidate points closer to instance centers. The final 3d bounding boxes are predicted on these generated candidate points. Relevant experiments on KITTI and nuScenes datasets verify that our proposed algorithm has achieved a competitive level compared with some state-of-the-art algorithms.

关键词： Feature extraction Three-dimensional displays Transformers Point cloud compression Convolution detection algorithms Correlation 3d object detection single-stage multi-source features transformer feature fusion point-wise features

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：