检索结果-内蒙古大学图书馆

Multimodal fusion via voting network for 3d object detection in indoors

PATTERN RECOGNITION 2025年 164卷

作者： Li, Jianxin Si, Guannan Liang, Xinyu An, Zhaoliang Tian, Pengxin Zhou, Fengyu Wang, Xiaoliang Shandong Jiaotong Univ Sch Elect Engn Informat Sci 5001 Haitang RdChangqing Univ Sci Pk Jinan 250300 Shandong Peoples R China Shandong Univ Sch Control Sci & Engn 17923 Jingshi Rd Jinan 250061 Shandong Peoples R China Shandong Longxihanzhang Technol Dev Co Ltd 607Unit 2Bldg 6Dinghao Plaza 44Gongye South Rd Jinan 250104 Shandong Peoples R China

In recent years, 3d object detection has become essential in machine vision systems, as it requires more spatial information, such as position and size, compared to traditional 2d detection. Numerous studies have successfully achieved accurate localization, size estimation, orientation estimation, and classification of objects in diverse scenarios. Building on this foundation, we propose a novel method called "Point Cloud and Image VoteNet," which enhances 3d object detection through the early fusion of radar point cloud data and image features. Our Point-Fusion technique projects radar point clouds onto images to extract complementary features. By incorporating point cloud density parameters, we improve the object matching mechanism, resulting in precise detections. Experimental results demonstrate that our model effectively leverages the combined information from point clouds and images, achieving superior performance. The fusion techniques and optimization strategies employed significantly enhance accuracy and robustness, showcasing promising potential in applications such as autonomous driving, robotics, and augmented reality.

关键词： 3d object detection Point cloud Image modalities deep learning techniques Multi-model fusion

来源：评论

学校读者我要写书评

暂无评论

CT3d++: Improving 3d object detection with Keypoint-Induced Channel-wise Transformer

引用

INTERNATIONAL JOURNAL OF COMPUTER VISION 2025年 1-20页

作者： Sheng, Hualian Cai, Sijia Zhao, Na deng, Bing Liang, Qiao Zhao, Min-Jian Ye, Jieping Alibaba Cloud Hangzhou 310052 Peoples R China Singapore Univ Technol & Design Informat Syst Technol & Design Pillar Singapore 487372 Singapore Zhejiang Univ Coll Informat Sci & Elect Engn Hangzhou 310027 Peoples R China

The field of 3d object detection from point clouds is rapidly advancing in computer vision, aiming to accurately and efficiently detect and localize objects in three-dimensional space. Current 3d detectors commonly fall short in terms of flexibility and scalability, with ample room for advancements in performance. In this paper, our objective is to address these limitations by introducing two frameworks for 3d object detection. Firstly, we propose CT3d, which sequentially performs raw-point-based embedding, a standard Transformer encoder, and a channel-wise decoder for point features within each proposal. Secondly, we present an enhanced network called CT3d++, which incorporates geometric and semantic fusion-based embedding to extract more valuable and comprehensive proposal-aware information. Additionally, CT3d++ utilizes a point-to-key bidirectional encoder for more efficient feature encoding with reduced computational cost. By replacing the corresponding components of CT3d with these novel modules, CT3d++ achieves state-of-the-art performance on both the KITTI dataset and the large-scale Waymo Open dataset.

关键词： 3d object detection Point clouds Transformer Geometric and semantic fusion

来源：评论

学校读者我要写书评

暂无评论

View-to-label: Multi-view consistency for self-supervised monocular 3d object detection

引用

COMPUTER VISION ANd IMAGE UNdERSTANdING 2025年 254卷

作者： Mouawad, Issa Brasch, Nikolas Manhardt, Fabian Tombari, Federico Odone, Francesca Univ Genoa DIBRIS MALGA Dibris Italy Tech Univ Munich Chirurg Munich Germany Google Inc Zurich Switzerland

For autonomous vehicles, driving safely is highly dependent on the capability to correctly perceive the environment in the 3d space, hence the task of 3d object detection represents a fundamental aspect of perception. While 3d sensors deliver accurate metric perception, monocular approaches enjoy cost and availability advantages that are valuable in a wide range of applications. Unfortunately, training monocular methods requires avast amount of annotated data. To compensate for this need, we propose a novel approach to self-supervise 3d object detection purely from RGB video sequences, leveraging geometric constraints and weak labels. Unlike other approaches that exploit additional sensors during training, our method relies on the temporal continuity of video sequences. A supervised pre-training on synthetic data produces initial plausible 3d boxes, then our geometric and photometrically grounded losses provide a strong self-supervision signal that allows the model to be fine-tuned on real data without labels. Our experiments on Autonomous driving benchmark datasets showcase the effectiveness and generality of our approach and the competitive performance compared to other self-supervised approaches.

关键词： 3d object detection Self-supervised learning Multi-view optimization Automotive application Monocular system

来源：评论

学校读者我要写书评

暂无评论

FogFusion: Robust 3d object detection based on camera-LidAR fusion for autonomous driving in foggy weather conditions

引用

PROCEEdINGS OF THE INSTITUTION OF MECHANICAL ENGINEERS PART d-JOURNAL OF AUTOMOBILE ENGINEERING 2025年

作者： Zhang, Bingli Wang, Yixin Zhang, Chengbiao Jiang, Junzhao Luo, Xiang Wang, Xinyu Zhang, Yangyang Liu, Zhongzheng Shen, Gan Ye, Yunsheng Jiang, Ping Hefei Univ Technol Sch Automot & Transportat Engn 193 Tunxi Rd Hefei 230009 Anhui Peoples R China Anhui Engn Lab Intelligent Automobile Hefei Peoples R China

Foggy environments present significant challenges to autonomous driving owing to the effects of attenuation and backscattering, which often compromise the performance of LidAR-camera fusion-based perception systems. In this study, we introduce FogFusion, a novel 3d object detection network specifically designed to operate effectively under foggy conditions by leveraging a synergistic camera-LidAR fusion approach. Our approach integrates a depth Completion network with Fog Convolution (dCFC) to generate virtual point clouds that enhance the original sparse LidAR data. These enhanced point clouds are then processed using a Flexible Cylindrical Voxel (FCV) encoding method. To ensure robust multi-modal feature integration, we employ a Cylindrical Fusion Module (CFM) during the fusion process. Experimental evaluations on the KITTI and KITTI-C datasets reveal that FogFusion improves detection performance in foggy conditions by at least 3.32% compared to the baseline model and surpasses the performance of advanced 3d object detection models. These results highlight FogFusion's potential to significantly enhance the environmental perception capabilities of autonomous vehicles operating in foggy weather conditions.

关键词： Autonomous vehicles 3d object detection multi-modal fusion perception in foggy weather conditions deep learning

来源：评论

学校读者我要写书评

暂无评论

AWARdistill: Adaptive and robust 3d object detection in adverse conditions through knowledge distillation

引用

EXPERT SYSTEMS WITH APPLICATIONS 2025年 266卷

作者： Liu, Yifan Zhang, Yong Lan, Rukai Cheng, Cheng Wu, Zhaolong Wuhan Univ Sci & Technol Sch Informat Sci & Engn Wuhan 430081 Peoples R China Minist Educ Engn Res Ctr Met Automat & Measurement Technol Wuhan 430081 Peoples R China Huazhong Univ Sci & Technol Sch Artificial Intelligence & Automat Wuhan 430074 Peoples R China

3d object detection is a crucial component of autonomous vehicle perception, but adverse weather conditions can affect sensor performance, leading to a deterioration in data quality, thereby posing significant challenges to the further development of perception. This paper presents a novel, robust 3d object detection framework to address this issue. Firstly, to tackle the problem of lacking adverse weather datasets, we propose the Multi- modal Adverse-Weather data Simulation Theory (MIST), which employs optical models to simulate fog and replicates the dynamic properties of rain and snow to recreate real-world circumstances. Secondly, we propose the Adaptive and Robust 3d object detection Framework in Adverse Conditions through Knowledge distillation (AWARdistill), which employs staged knowledge distillation to enable the model to adapt to adverse weather conditions, significantly enhancing detection accuracy and robustness. Additionally, we designed two modules that can be integrated into other detection frameworks to enhance robustness. We evaluated the performance of AWARdistill on multiple datasets. On the KITTI dataset, our model attained an average precision of about 88% and can efficiently adapt to extreme weather. Extensive experiments demonstrate our model's effectiveness and superiority, providing strong support for autonomous driving in challenging weather environments.

关键词： Autonomous driving 3d object detection Adverse weather Knowledge distillation

来源：评论

学校读者我要写书评

暂无评论

FSd V2: Improving Fully Sparse 3d object detection With Virtual Voxels

引用

IEEE TRANSACTIONS ON PATTERN ANALYSIS ANd MACHINE INTELLIGENCE 2025年第2期47卷 1279-1292页

作者： Fan, Lue Wang, Feng Wang, Naiyan Zhang, Zhaoxiang Chinese Acad Sci CASIA Inst Automat New Lab Pattern Recognit NLPR State Key Lab Multimodal Artificial Intelligence S Beijing 100190 Peoples R China TuSimple Beijing 100020 Peoples R China

LidAR-based fully sparse architecture has gained increasing attention. FSdv1 stands out as a representative work, achieving impressive efficacy and efficiency, albeit with intricate structures and handcrafted designs. In this paper, we present FSdv2, an evolution that aims to simplify the previous FSdv1 and eliminate the ad-hoc heuristics in its handcrafted instance-level representation, thus promoting better universality. To this end, we introduce virtual voxels, taking over the clustering-based instance segmentation in FSdv1. Virtual voxels not only address the notorious issue of the Center Feature Missing in fully sparse detectors but also endow the framework with a more elegant and streamlined approach. Besides, we develop a suite of components to complement the virtual voxel mechanism, including a virtual voxel encoder, a virtual voxel mixer, and a virtual voxel assignment strategy. We conduct experiments on three large-scale datasets: Waymo Open dataset, Argoverse 2 dataset, and nuScenes dataset. Our results showcase state-of-the-art performance on all three datasets, highlighting the superiority of FSdv2 in long-range scenarios and its universality in achieving competitive performance across diverse scenarios. Moreover, we provide comprehensive experimental analysis to understand the workings of FSdv2.

关键词： Feature extraction detectors Three-dimensional displays object detection Mixers Instance segmentation Point cloud compression Convolution Computer architecture Pipelines 3d object detection point cloud LidAR sparse Waymo Open dataset nuScenes dataset Argoverse 2 dataset

来源：评论

学校读者我要写书评

暂无评论

Cross-Supervised LidAR-Camera Fusion for 3d object detection

引用

IEEE ACCESS 2025年 13卷 10447-10458页

作者： Zuo, Chao Jie Gu, Cao Yu Guo, Yi Kun Miao, Xiao dong Nanjing Tech Univ Sch Mech & Power Engn Nanjing 211889 Peoples R China

Fusing LidAR and camera information is essential for achieving accurate and reliable 3d object detection in autonomous driving systems. due to inherent differences between different modalities, seeking an efficient and accurate fusion method is of great importance. Recently, significant progress has been made in 3d object detection methods based on lift-splat-shot (LSS-based) approaches. However, inaccurate depth estimation and substantial semantic information loss remain significant factors limiting the accuracy of 3d detection. In this paper, we propose a cross-fusion framework under a dual spatial representation, by integrating information in different spatial representations, namely bird's-eye view (BEV) and camera view, and establishing soft links to fully utilize the information carried by different modalities. It consists of two important components, gated LidAR supervised BEV (GLS-BEV) and multi-attention cross fusion (MACF) modules. The former achieves accurate depth estimation by supervising the transformation of LidAR data with clear depth into the image space, constructing point cloud features in vehicle's perspective. The latter utilizes three sub-attention modules with different roles to achieve cross-modal interaction within the same space, effectively reducing semantic loss. On the nuScenes benchmark, our proposed method achieves outstanding 3d object detection results with 71.8 mAP and 74.2 NdS.

关键词： Feature extraction Three-dimensional displays Laser radar Point cloud compression Cameras object detection Semantics Accuracy Logic gates depth measurement 3d object detection LidAR-camera system multi-sensor fusion BEV

来源：评论

学校读者我要写书评

暂无评论

NCFdet: Enhanced point cloud features using the neural collapse phenomenon in multimodal fusion for 3d object detection

引用

JOURNAL OF COMPUTATIONAL dESIGN ANd ENGINEERING 2025年第1期12卷 300-311页

作者： Xu, Yaming Xu, Minglei Wang, Yan Li, Boliang Harbin Inst Technol Sch Astronaut Harbin 150000 Peoples R China

The accurate and effective detection of 3d objects represents a crucial component in the development of multi-sensor autonomous driving and robotics systems, particularly in the context of navigating complex urban environments. The complementary nature of image and point cloud data allows for the detection of objects with greater accuracy and robustness when both image and point cloud features are employed. At present, there is no optimal solution for the timing of multi-sensor fusion, particularly in the case of cross-modal data formats. In order to address these issues, we propose a multi-sensor object detection scheme based on Neural Collapse (NC) theory augmented point cloud, which we have designated NCFdet. In particular, we have incorporated an image pre-training model at the network layer where the NC occurs and designed a fusion module based on Transformer Attention. Furthermore, we investigate the interconnection between NC and transmodal transfer, and provide an explanation for the efficacy of the former based on the latter. The NCFdet system performs well for the detection of small targets in scenes due to the advantage of a compact image data format. The performance of the proposed NCFdet system was validated on the KITTI (Karlsruhe Institute of Technology and Toyota Technological Institute) dataset in comparison to existing methods. Furthermore, the object detection performance of the proposed framework was demonstrated on our visual rotation platform. The results show that the detection system achieves advanced fusion results. The code will be published after the paper is published.

关键词： 3d object detection multi-sensor systems neural collapse feature fusion

来源：评论

学校读者我要写书评

暂无评论

CFPC: The Curbed Fake Point Collector to Pseudo-LidAR-Based 3d object detection for Autonomous Vehicles

引用

IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY 2025年第2期74卷 1922-1934页

作者： Gao, Honghao Shao, Jie Iqbal, Muddesar Wang, Ye Xiang, Zhengzhe Shanghai Univ Sch Comp Engn & Sci Shanghai 200444 Peoples R China Prince Sultan Univ Coll Engn Renewable Energy Lab Riyadh 11586 Saudi Arabia Hangzhou City Univ Sch Comp & Computat Sci Hangzhou 310015 Peoples R China

3d object detection in autonomous driving systems perceives the surrounding environment and is the foundation for autonomous driving. due to the sparsity inherent in point clouds in autonomous driving scenarios, LidAR-based 3d object detection often fails to distinguish distant objects effectively. Addressing the issue of point cloud sparsity will enhance the detection range in autonomous driving scenarios. Pseudo point clouds have been used to enhance the ability of deep learning models to detect distant points. However, this approach has several shortcomings. In this paper, a curbed fake point collector (CFPC), which addresses the three issues caused by pseudo points, is proposed to support 3d object detection for autonomous vehicles. First, for noise points with inaccurate coordinates, the dead pixel checker (dPC) calculates the depth map gradient using the Sobel operator. This approach enables the deep learning model to identify noise points. Second, because of the excessive quantity of points, sparse prioritized local sampling (SPLS) reduces the number of input point clouds to a lightweight level that can be accommodated by computing devices with limited memory. This is achieved through grid-based random sampling and real-point-prioritized farthest point sampling. This module effectively samples an appropriate pseudo point cloud based on the density of points in local space. Third, with respect to interference among channels, channel mask set abstraction (CMSA) isolates channels describing different information within the point cloud using GroupMLP, which is an MLP that separates channels into their respective groups. Group separation facilitates the extraction of features without mutual influence, allocating half of the output channels to color information and the other half to geometric information. The effectiveness of our approach is demonstrated by the results of experiments conducted on the KITTI dataset. It is superior to the baseline in most situations,

关键词： Point cloud compression Feature extraction Three-dimensional displays object detection Autonomous vehicles Semantics object recognition 3d object detection autonomous driving grid-based down sampling multimodality pseudo-LidAR set abstraction

来源：评论

学校读者我要写书评

暂无评论

Enhanced frustrum multi-scale VoteNet for 3d object detection in cluttered indoor scene

引用

APPLIEd INTELLIGENCE 2025年第7期55卷 1-15页

作者： Zhang, Xuesong He, Yu Song, Cunli Zhuang, Yan Dalian Jiaotong Univ Sch Railway Intelligent Engn Dalian 116028 Liaoning Peoples R China Dalian Univ Technol Sch Control Sci & Engn Dalian 116024 Liaoning Peoples R China

The low-cost Kinect sensor is capable of simultaneously acquiring RGB images and depth information, providing a comprehensive representation of 3d scenes. However, a significant drawback of the Kinect sensor is its susceptibility to low perception accuracy, particularly in cluttered indoor scenes. To tackle these issues, we introduce a novel EF-MSVoteNet framework for Kinect-based indoor 3d object detection. This framework integrates two key modules: The Enhanced Frustum (EF) and Multi-Scale Voting Network (MSVoteNet). After obtaining 2d bounding boxes via a 2d detector, the EF module no longer segments the scene but instead accumulates all frustum regions within a three-dimensional space. The EF module not only enhances the resolution of point clouds within the frustum in relation to the background but also substantially enhances the feature representation of object-related point clouds within the 3d scene. In addition, the proposed MSVoteNet module is a structurally flexible multi-scale voting network. It enhances feature extraction and integration across different scales by incorporating a multi-scale structure into the traditional VoteNet. The performance analysis is carried out on the SUN RGB-d and ScanNet datasets, which were collected using RGB-d sensors in cluttered indoor environments, and it demonstrates the efficacy and effectiveness of our proposed methods. The source code is available at: https://***/zerrows2/EF-MSVoteNet

关键词： 3d object detection Cluttered indoor scene Frustum Hough voting

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：