检索结果-内蒙古大学图书馆

MENet: Multi-Modal Mapping Enhancement Network for 3d object detection in Autonomous driving

IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS 2024年第8期25卷 9397-9410页

作者： Liu, Moyun Chen, Youping Xie, Jingming Zhu, Yijie Zhang, Yang Yao, Lei Bing, Zhenshan Zhuang, Genghang Huang, Kai Zhou, Joey Tianyi Huazhong Univ Sci & Technol Sch Mech Sci & Engn Wuhan 430074 Peoples R China ASTAR IHPC Singapore 138632 Singapore ASTAR CFAR Singapore 138632 Singapore Hubei Univ Technol Sch Mech Engn Wuhan 430068 Peoples R China Nanjing Univ Natl Key Lab Novel Software Technol Nanjing 210023 Peoples R China Hong Kong Polytech Univ Dept Elect & Elect Engn Hong Kong Peoples R China Tech Univ Munich Dept Informat D-85748 Munich Germany Sun Yat Sen Univ Sch Data & Comp Sci Guangzhou 510006 Peoples R China

To achieve more accurate perception performance, LidAR and camera are gradually chosen to improve 3d object detection simultaneously. However, it is still a non-trivial task to build an effective fusion mechanism, and this is hindering the development of multi-modal based method. Especially, the mapping relationship construction between two modalities is far from fully explored. Canonical cross-modal mapping suffers from failure when the calibration matrix is incorrect, and it also greatly wastes the amount and density of RGB image information. This paper aims to extend the traditional one-to-one alignment relationship between LidAR and camera. For all projected point clouds, we enhance their cross-modal mapping relationship through aggregating color-texture related feature and shape-contour related feature. Further, a mapping pyramid is proposed to leverage the semantic representation of the image feature at different stages. Based on the above mapping enhancement strategies, our method increases the engagement rate of image. Finally, we design a fusion module based on an attention mechanism to improve the point cloud feature with the auxiliary image feature. Extensive experiments on the KITTI dataset and SUN-RGBd dataset show that our model achieves satisfactory 3d object detection, especially for categories with sparse point clouds compared with other multi-modal fusion networks.

关键词： Multi-modal fusion 3d object detection mapping enhancement autonomous driving

来源：评论

学校读者我要写书评

暂无评论

PCdR-dFF: multi-modal 3d object detection based on point cloud diversity representation and dual feature fusion

引用

NEURAL COMPUTING & APPLICATIONS 2024年第16期36卷 9329-9346页

作者： Xia, Chenxing Li, Xubing Gao, Xiuju Ge, Bin Li, Kuan-Ching Fang, Xianjin Zhang, Yan Yang, Ke Anhui Univ Sci & Technol Coll Comp Sci & Engn Huainan 232001 Peoples R China Inst Energy Hefei Comprehens Natl Sci Ctr Hefei Anhui Peoples R China Anhui Purvar Bigdata Technol Co Ltd Huainan 232001 Peoples R China Anhui Univ Sci & Technol Coll Elect & Informat Engn Huainan Anhui Peoples R China Providence Univ Dept Comp Sci & Informat Engn Taichung Taiwan Inst Artificial Intelligence Hefei Comprehens Natl Sci Ctr Hefei Peoples R China Anhui Univ Sch Elect & Informat Engn Hefei Anhui Peoples R China

Recently, multi-modal 3d object detection techniques based on point clouds and images have received increasing attention. However, existing methods for multi-modal feature fusion are often relatively singular, and single point cloud representation methods also have some limitations. For example, voxelization may result in the loss of fine-grained information, while 2d images lack depth information, which can restrict the accuracy of detection. Therefore, in this work, we propose a novel method for multi-modal 3d object detection based on point cloud diversity representation and dual feature fusion, PCdR-dFF, to improve the prediction accuracy of 3d object detection. Firstly, point clouds are projected to the image coordinate system and extract multi-level features of the point cloud corresponding to the image using a 2d backbone network. Then, the point clouds are jointly characterized using graphs and pillars, and the 3d features of the point clouds are extracted using graph neural networks and residual connectivity. Finally, a dual feature fusion method is designed to improve the accuracy of detection with the help of a well-designed multi-point fusion model and multi-feature fusion mechanism embedded with a spare 3d-U Net. Extensive experiments on the KITTI dataset demonstrate the effectiveness and competitiveness of our proposed models in comparison with other methods.

关键词： 3d object detection Graph neural networks Multi-modal Point cloud

来源：评论

学校读者我要写书评

暂无评论

Focal-PETR: Embracing Foreground for Efficient Multi-Camera 3d object detection

IEEE TRANSACTIONS ON INTELLIGENT VEHICLES

引用

IEEE TRANSACTIONS ON INTELLIGENT VEHICLES 2024年第1期9卷 1481-1489页

作者： Wang, Shihao Jiang, Xiaohui Li, Ying Beijing Inst Technol Beijing 100081 Peoples R China

The dominant multi-camera 3d detection paradigm is based on explicit 3d feature construction, which requires complicated indexing of local image-view features via 3d-to-2d projection. Other methods implicitly introduce geometric positional encoding and perform global attention (e.g., PETR) to build the relationship between image tokens and 3d objects. The 3d-to-2d perspective inconsistency and global attention lead to a weak correlation between foreground tokens and queries, resulting in slow convergence. We propose Focal-PETR with instance-guided supervision and spatial alignment module to adaptively focus object queries on discriminative foreground regions. Focal-PETR additionally introduces a down-sampling strategy to reduce the consumption of global attention. Our model achieves leading performance on the large-scale nuScenes benchmark and a superior speed of 30 FPS on a single RTX3090 GPU. Extensive experiments show that our method outperforms PETR while consuming 3x fewer training hours. The code is made publicly available.

关键词： Three-dimensional displays Feature extraction Training Transformers object detection Cameras decoding 3d object detection Autonomous driving detection Transformer

来源：评论

学校读者我要写书评

暂无评论

dual-domain deformable feature fusion for multi-modal 3d object detection

引用

JOURNAL OF ELECTRONIC IMAGING 2024年第6期33卷

作者： Wang, Shihao deng, Tao Chongqing Jiaotong Univ Sch Mechatron & Vehicle Engn Chongqing Peoples R China Chongqing Jiaotong Univ Sch Aeronaut Chongqing Peoples R China Chongqing Key Lab Green Aviat Energy & Power Chongqing Peoples R China Chongqing Jiaotong Univ Green Aerotech Res Chongqing Peoples R China

Recent advancements in 3d object detection using light detection and ranging (LidAR)-camera fusion have enhanced autonomous driving perception. However, aligning LidAR and image data during multimodal fusion remains a significant challenge. We propose a novel multi-modal feature alignment and fusion architecture to effectively align and fuse voxel and image data. The proposed architecture comprises four key modules. Z-axis attention aggregates voxel features along the vertical axis using self-attention. Voxel-domain deformable encoder improves context understanding with deformable attention to encode voxel features. dual-domain deformable feature alignment uses deformable attention to adaptively align voxel and image features, addressing resolution mismatches. Finally, gated fusion utilizes a gating mechanism to dynamically fuse aligned features. The multi-layer design further enhances feature detail retention and improves dual-domain fusion performance. Experimental results show our method increases average precision by 2.41% at the "hard" difficulty level for cars on the KITTI test set. On the KITTI validation set, mean average precision improves by 1.06% for cars, 6.88% for pedestrians, and 1.83% for cyclists. (c) 2024 SPIE and IS&T

关键词： 3d object detection multi-modal fusion feature alignment deformable attention autonomous driving

来源：评论

学校读者我要写书评

暂无评论

BEVRefiner: Improving 3d object detection in Bird's-Eye-View via dual Refinement

引用

IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS 2024年第10期25卷 15094-15105页

作者： Wang, Binglu Zheng, Haowen Zhang, Lei Liu, Nian Anwer, Rao Muhammad Cholakkal, Hisham Zhao, Yongqiang Li, Zhijun Northwestern Polytech Univ Sch Automat Xian 710072 Peoples R China Beijing Inst Technol Radar Res Lab Natl Key Lab Sci & Technol Space Born Intelligent Beijing 100811 Peoples R China Beijing Inst Technol Sch Informat & Elect Beijing 100811 Peoples R China Mohamed bin Zayed Univ Artificial Intelligence Comp Vis Dept Abu Dhabi U Arab Emirates Tongji Univ Sch Mech Engn Shanghai 200070 Peoples R China

Many multi-view camera-based 3d object detection models transform the image features into Bird's-Eye-View (BEV) via the Lift-Splat-Shoot (LSS) mechanism, which "lifts" 2d camera-view features to the 3d voxel space based on the predicted depth distribution and then "splats" 3d features into a BEV plane for subsequent 3d object detection. However, the BEV feature in such a one-stage view transformation scheme heavily relies on the quality of the predicted depth distribution and 2d camera-view features, which further determines the final detection performance. In this paper, we propose a BEVRefiner model which performs dual refinement for both depth prediction and 2d camera-view features. On the one hand, we perform light-weight depth refinement in the depth distribution frustum space by incorporating 3d context and depth distribution prior. On the other hand, we reproject the BEV feature back to each camera view to enhance 2d image features. In this way, the original camera-view features can be enhanced by implicitly incorporating 3d contexts and multi-view contexts, which cannot be achieved in the original 2d camera view. We also propose to use dominant depth bins only for the reprojection to save computational burden. Finally, we generate the refined BEV feature using the refined depth distribution and camera-view features for more accurate 3d object detection. Our BEVRefiner can be plugged into LSS-based BEV detectors and we perform extensive experiments on the representative model BEVdet, which strongly verified the efficiency of our proposed approach under several settings.

关键词： 3d object detection BEV depth prediction refinement

来源：评论

学校读者我要写书评

暂无评论

Robustness-Aware 3d object detection in Autonomous driving: A Review and Outlook

引用

IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS 2024年第11期25卷 15407-15436页

作者： Song, Ziying Liu, Lin Jia, Feiyang Luo, Yadan Jia, Caiyan Zhang, Guoxin Yang, Lei Wang, Li Beijing Jiaotong Univ Sch Comp Sci & Technol Beijing Key Lab Traff Data Anal & Min Beijing 100044 Peoples R China Univ Queensland Sch Informat Technol & Elect Engn St Lucia Qld 4072 Australia Beijing Univ Posts & Telecommun Sch Comp Sci Beijing 100876 Peoples R China Tsinghua Univ State Key Lab Intelligent Green Vehicle & Mobil Beijing 100084 Peoples R China Tsinghua Univ Sch Vehicle & Mobil Beijing 100084 Peoples R China Beijing Inst Technol Sch Mech Engn Beijing 100081 Peoples R China

In the realm of modern autonomous driving, the perception system is indispensable for accurately assessing the state of the surrounding environment, thereby enabling informed prediction and planning. The key step to this system is related to 3d object detection that utilizes vehicle-mounted sensors such as LidAR and cameras to identify the size, the category, and the location of nearby objects. despite the surge in 3d object detection methods aimed at enhancing detection precision and efficiency, there is a gap in the literature that systematically examines their resilience against environmental variations, noise, and weather changes. This study emphasizes the importance of robustness, alongside accuracy and latency, in evaluating perception systems under practical scenarios. Our work presents an extensive survey of camera-only, LidAR-only, and multi-modal 3d object detection algorithms, thoroughly evaluating their trade-off between accuracy, latency, and robustness, particularly on datasets like KITTI-C and nuScenes-C to ensure fair comparisons. Among these, multi-modal 3d detection approaches exhibit superior robustness, and a novel taxonomy is introduced to reorganize the literature for enhanced clarity. This survey aims to offer a more practical perspective on the current capabilities and the constraints of 3d object detection algorithms in real-world applications, thus steering future research towards robustness-centric advancements.

关键词： 3d object detection perception robustness autonomous driving

来源：评论

学校读者我要写书评

暂无评论

A Smart IoT Enabled End-to-End 3d object detection System for Autonomous Vehicles

引用

IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS 2023年第11期24卷 13078-13087页

作者： Ahmed, Imran Jeon, Gwanggil Chehri, Abdellah Anglia Ruskin Univ Sch Comp & Informat Sci Cambridge CB1 1PT England Incheon Natl Univ Dept Embedded Syst Engn Incheon 22012 South Korea Royal Mil Coll Canada Dept Math & Comp Sci Kingston ON K7K 7B4 Canada

Integration of advanced signal processing, image processing, deep learning, edge computing, and the Internet of Things (IoT) into vehicles allows intelligent automated vehicles to navigate autonomously in different environments. It is crucial for reliable and safe driving that an autonomous vehicle can accurately, effectively, and efficiently recognize, perceive, and observe the surrounding environments. Autonomous vehicles comprise advanced sensor technologies such as RGB cameras and LidaR that produce an extensive data set in the form of RGB images and 3d measurement points, also recognized as a point cloud. It is necessary to understand and interpret collected data information efficiently and to identify other road users, such as pedestrians and vehicles. Thus, we introduced a smart IoT-enabled deep learning based end-to-end 3d object detection system that works in real-time, emphasizing autonomous driving situations. The detection model is based on YOLOv3;firstly, the model is utilized for 2d object detection and then modified for 3d object detection purposes. The presented model uses point cloud, and RGB image data as input and outputs detected bounding boxes with confidence scores and class labels. Experiments are carried out on the Lyft data set;results reveal that the YOLOv3 model achieves high accuracy and outperforms from other state-of-the-art detection models in terms of effectiveness and accuracy. The overall accuracy of the model is 96% and 97% for 2d and 3d object detection, respectively.

关键词： Autonomous vehicles Internet of Things digital twins deep learning 3d object detection YOLOv3 lyft

来源：评论

学校读者我要写书评

暂无评论

Adaptive depth position encoding for sparse query-based 3d object detection from multi-camera images

引用

PATTERN RECOGNITION 2025年 167卷

作者： Linghu, Junjie Ling, Qiang Univ Sci & Technol China Hefei 230027 Peoples R China

Recently, multi-camera based 3d object detection has found wide applications in the field of autonomous driving. To accomplish that task, dense query-based detectors build explicit dense BEV(Bird's Eye View) features and execute complex view transformations between the image and BEV spaces, resulting in high computational cost. In contrast, sparse query-based detectors directly utilize object queries to perform prediction, but suffer from sub-optimal performance. To achieve superior performance without incurring high computational cost and prolonged inference time, we propose a sparse query-based 3d object detector, named AdPE (Adaptive depth Position Encoding). Specifically, we develop an adaptive depth generator to produce precise depth maps, which are utilized to encode positional information in the 3d space. To further boost efficacy, we enhance AdPE with temporal fusion through updating a memory bank to store historical information and designing a style attention module to implicitly model objects' movements. Comprehensive experiments on the nuScenes dataset demonstrate that AdPE significantly improves 3d object detection performance compared with some state-of-the-art detectors. The codes of AdPE are available at: https://***/ustc-kkkkk/AdPE.

关键词： Autonomous driving 3d object detection Multi-camera images 3d position encoding

来源：评论

学校读者我要写书评

暂无评论

NRAP-RCNN: A Pseudo Point Cloud 3d object detection Method Based on Noise-Reduction Sparse Convolution and Attention Mechanism

引用

INFORMATION 2025年第3期16卷 176-176页

作者： Zhou, Ziyue Jia, Yongqing Zhu, Tao Wan, Yaping Univ South China Sch Comp Sci Hengyang 421001 Peoples R China Hunan Elect Res Inst Testing Grp Co Ltd Technol Dept Xiangxiang 411402 Peoples R China

In recent years, pseudo point clouds generated from depth completion of RGB images and LidAR data have provided a robust foundation for multimodal 3d object detection. However, the generation process often introduces noise, reducing data quality and detection accuracy. Moreover, existing methods fail to effectively capture channel correlations and global contextual information during the 2d feature extraction stage after the 3d backbone network, limiting detection performance. To address these challenges, this paper proposes NRAP-RCNN, a pseudo point cloud-based 3d object detection method with two key innovations: (1) A noise-reduction sparse convolution network (NRConvNet), comprising NRConv (noise-resistant submanifold sparse convolution), SRB (sparse convolution residual block), and MHSA (multi-head self-attention). NRConv suppresses pseudo point cloud noise by jointly encoding 2d and 3d features, SRB enhances feature extraction depth and robustness, and MHSA optimizes global feature representation. (2) An attention fusion module (ECA_GCA) is introduced to enhance the feature representation of the 2d backbone network by combining channel and global contextual information. The experimental results demonstrate that NRAP-RCNN achieves 88.4% car AP (R40) on the KITTI validation set and 85.1% on the test set, significantly outperforming advanced 3d detection methods, showcasing its effectiveness in improving detection performance.

关键词： pseudo point cloud 3d object detection noise reduction sparse convolution attention mechanism

来源：评论

学校读者我要写书评

暂无评论

Pillar-X: Integrating Self-Learned Image Features to Improve 3d object detection

引用

IEEE ACCESS 2025年 13卷 83071-83081页

作者： Csontho, Mihaly Rovid, Andras Budapest Univ Technol & Econ Fac Transportat Engn & Vehicle Engn Dept Automot Technol H-1111 Budapest Hungary

Accurate 3d object detection is essential for robust perception systems in autonomous vehicles. This paper presents Pillar-X, a 3d object recognition framework designed to generate and use self-learned image features. Unlike approaches that rely on pre-trained networks, Pillar-X efficiently learns and fuses features from an image directly within an end-to-end pipeline. Comprehensive evaluations on the KITTI and CARLA-generated dataset demonstrate Pillar-X's significant reduction in false positive detections, especially for the pedestrian and cyclist classes. The proposed model achieves this while maintaining a speed similar to the baseline algorithm (> 20 Hz). By comparing the network using both PASCAL and a specific criterion, it can be concluded that Pillar-X is able to improve accuracy and reliability in 3d object detection tasks.

关键词： Pillar-X fusion 3d object detection KITTI CARLA-generated dataset Pillar-X fusion 3d object detection KITTI CARLA-generated dataset

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：