检索结果-内蒙古大学图书馆

MonoMPV: Monocular 3d object detection With Multiple Projection Views on Edge devices

IEEE ACCESS 2024年 12卷 136599-136612页

作者： deng, Zhaoxue Hao, Bingsen Liu, Guofang Li, Xingquan Wei, Hanbing Huang, Fei Liu, Shengshu Chongqing Jiaotong Univ Sch Mechatron & Vehicle Engn Chongqing 400074 Peoples R China Chongqing Changan Automobile Co Ltd Res & Dev Dept Chongqing 400023 Peoples R China China Soc Automot Engineers Beijing 100176 Peoples R China Chongqing Univ Technol Vehicle Engn Inst Chongqing 400054 Peoples R China China Rd & Bridge Corp Beijing 100010 Peoples R China

In the field of autonomous driving, monocular 3d object detection is focused on the task of representing 3d scenes using a single camera image and conducting 3d object detection. While Bird's-Eye View (BEV) method effectively decreases the computational burden associated with 3d scene representation, its limitation in considering height information can lead to a less accurate depiction of complex 3d structures. This study introduces an innovative monocular 3d object detection framework called MonoMPV. This framework represents a complete 3d scene by mapping spatial objects onto Multi-Projection Views (MPV) without the need for voxelization, thus simplifying the process. Notably, MPV systems consist of Feature Cross-Attention (FCA) and Projection Cross-Attention (PCA) components. FCA aims to enhance image features to MPV level, while PCA enables direct information interaction among the views within MPV. Furthermore, Triplet Loss for Top Feature (TLTF) was employed in conjunction with FCA and PCA to distinguish effectively between top-plane and background features. By engaging in this practice, it is possible to develop more complex 3d structural models and establish precise optimization objectives through TLTF. Consequently, this approach enhances the effective utilization of data by the model. Experimental results on the nuScenes dataset illustrate that this approach surpasses existing monocular 3d object detection methods. To implement algorithms on on-board edge computing devices, the monocular 3d object detection task has been executed on the edge device Jetson Orin NX, ensuring high precision.

关键词： Three-dimensional displays object detection Feature extraction Computational modeling Autonomous vehicles Accuracy Solid modeling Autonomous driving 3d object detection multi-projection views edge computing

来源：评论

学校读者我要写书评

暂无评论

HRNet: 3d object detection network for point cloud with hierarchical refinement

引用

PATTERN RECOGNITION 2024年 149卷

作者： Lu, Bin Sun, Yang Yang, Zhenyu Song, Ran Jiang, Haiyan Liu, Yonghuai North China Elect Power Univ Dept Comp Baoding 071003 Hebei Peoples R China Shandong Univ Sch Control Sci & Engn Jinan 250100 Shandong Peoples R China Nanjing Agr Univ Coll Artificial Intelligence Nanjing 210095 Jiangsu Peoples R China Edge Hill Univ St Helens Rd Ormskirk L39 4QP Lancs England

Recently, 3d object detection from LidAR point clouds has advanced rapidly. Although the second stage can improve the detection performance significantly, prior works concern little about the essential differences among different stages for the performance enhancement. To address this, this paper proposes a Hierarchical Refinement Network (HRNet) with two novel strategies. Firstly, we build the detection head on multiscale voxel features to optimize the regression branch progressively with an effective Scale-aware Attentive Propagation (SAP) module. Then, we propose a dynamic Sample Selection (dSS) module for the recalculation of the IoU during each stage to obtain more balanced positive and negative sample selections. Experiments over benchmark datasets show the effectiveness of our HRNet, particularly for car detection in the sparse point clouds.

关键词： 3d object detection LidAR point clouds Multi-scale features Transformer dynamic sample selection Hierarchical refinemen

来源：评论

学校读者我要写书评

暂无评论

SRFdet3d: Sparse Region Fusion based 3d object detection

引用

NEUROCOMPUTING 2024年 593卷

作者： Erabati, Gopi Krishna Araujo, Helder Univ Coimbra Inst Syst & Robot Rua Silvio Lima Polo 2 P-3030290 Coimbra Portugal

Unlike the earlier 3d object detection approaches that formulate hand-crafted dense (in thousands) object proposals by leveraging anchors on dense feature maps, we formulate np (in hundreds) number of learnable sparse object proposals to predict 3d bounding box parameters. The sparse proposals in our approach are not only learnt during training but also are input-dependent, so they represent better object candidates during inference. Leveraging the sparse proposals, we fuse only the sparse regions of multi-modal features and we propose S parse R egion F usion based 3d object det ection (SRFdet3d) network with mainly three components: an encoder for feature extraction, a region proposal generation module for sparse input-dependent proposals and a decoder for multi-modal feature fusion and iterative refinement of object proposals. Additionally for optimal training, we formulate our sparse detector with many-to-one label assignment based on Optimal Transport Algorithm (OTA). We conduct extensive experiments and analysis on publicly available large-scale autonomous driving datasets: nuScenes, KITTI, and Waymo. Our LidAR-only SRFdet3d-L network achieves 63.1 mAP and outperforms the state-of-the-art networks on the nuScenes dataset, surpassing the dense detectors on KITTI and Waymo datasets. Our LidAR-Camera model SRFdet3d achieves 64.7 mAP with improvements over existing fusion methods.

关键词： 3d object detection Fusion Camera LidAR Autonomous driving Computer vision

来源：评论

学校读者我要写书评

暂无评论

Selective Transfer Learning of Cross-Modality distillation for Monocular 3d object detection

引用

IEEE TRANSACTIONS ON CIRCUITS ANd SYSTEMS FOR VIdEO TECHNOLOGY 2024年第10期34卷 9925-9938页

作者： ding, Rui Yang, Meng Zheng, Nanning Xi An Jiao Tong Univ Inst Artificial Intelligence & Robot Xian 710049 Peoples R China

Monocular 3d object detection is a promising yet ill-posed task for autonomous vehicles due to the lack of accurate depth information. Cross-modality knowledge distillation could effectively transfer depth information from LidAR to image-based network. However, modality gap between image and LidAR seriously limits its accuracy. In this paper, we systematically investigate the negative transfer problem induced by modality gap in cross-modality distillation for the first time, including not only the architecture inconsistency issue but more importantly the feature overfitting issue. We propose a selective learning approach named MonoSTL to overcome these issues, which encourages positive transfer of depth information from LidAR while alleviates the negative transfer on image-based network. On the one hand, we utilize similar architectures to ensure spatial alignment of features between image-based and LidAR-based networks. On the other hand, we develop two novel distillation modules, namely depth-Aware Selective Feature distillation (dASFd) and depth-Aware Selective Relation distillation (dASRd), which selectively learn positive features and relationships of objects by integrating depth uncertainty into feature and relation distillations, respectively. Our approach can be seamlessly integrated into various CNN-based and dETR-based models, where we take three recent models on KITTI and a recent model on NuScenes for validation. Extensive experiments show that our approach considerably improves the accuracy of the base models and thereby achieves the best accuracy compared with all recently released SOTA models. The code is released on https://***/dingCodeLab/MonoSTL.

关键词： Three-dimensional displays Laser radar Uncertainty object detection Feature extraction Estimation Knowledge engineering 3d object detection depth estimation cross-modality knowledge distillation selective transfer

来源：评论

学校读者我要写书评

暂无评论

Survey and systematization of 3d object detection models and methods

引用

VISUAL COMPUTER 2024年第3期40卷 1867-1913页

作者： drobnitzky, Moritz Friederich, Jonas Egger, Bernhard Zschech, Patrick Tech Univ Dresden Munchner Pl 3 D-01187 Dresden Germany Univ Southern Denmark Maersk McKinney Moller Inst Campusvej 55 DK-5230 Odense Denmark Friedrich Alexander Univ Erlangen Nurnberg Schlosspl 4 D-91054 Erlangen Germany

Strong demand for autonomous vehicles and the wide availability of 3d sensors are continuously fueling the proposal of novel methods for 3d object detection. In this paper, we provide a comprehensive survey of recent developments from 2012-2021 in 3d object detection covering the full pipeline from input data, over data representation and feature extraction to the actual detection modules. We introduce fundamental concepts, focus on a broad range of different approaches that have emerged over the past decade, and propose a systematization that provides a practical framework for comparing these approaches with the goal of guiding future development, evaluation, and application activities. Specifically, our survey and systematization of 3d object detection models and methods can help researchers and practitioners to get a quick overview of the field by decomposing 3dOd solutions into more manageable pieces.

关键词： 3d object detection 6-doF Tracking Survey

来源：评论

学校读者我要写书评

暂无评论

MMFG: Multimodal-based Mutual Feature Gating 3d object detection

引用

JOURNAL OF INTELLIGENT & ROBOTIC SYSTEMS 2024年第2期110卷 85-85页

作者： Xu, Wanpeng Fu, Zhipeng Peng Cheng Lab Dept New Pattern Network Xingke 1st St Shenzhen 518055 Guangdong Peoples R China

To address the problem that image and point cloud features are fused in a coarse fusion way and cannot achieve deep fusion, this paper proposes a multimodal 3d object detection architecture based on a mutual feature gating mechanism. First, since the feature aggregation approach based on the set abstraction layer cannot obtain fine-grained features, a point-based self-attention mechanism module is designed. This module is added to the extraction branch of point cloud features to achieve fine-grained feature aggregation while maintaining accurate location information. Second, a new gating mechanism is designed for the deep fusion of image and point cloud. deep fusion is achieved by mutual feature weighting between the image and the point cloud. The newly fused features are then fed into a feature refinement network to extract classification confidence and 3d target bounding boxes. Finally, a multi-scale detection architecture is proposed to obtain a more complete object shape. The location-based encoding feature algorithm is also designed to focus the interest points in the region of interest adaptively. The whole architecture shows outstanding performance on the KITTI3d and nuSenece datasets, especially at the difficult level. It shows that the framework solves the problem of low detection rates in LidAR mode due to the low number of surface points obtained from distant objects.

关键词： 3d object detection LidAR Multimodal fusion Gating mechanism

来源：评论

学校读者我要写书评

暂无评论

Scalable 3d object detection Pipeline With Center-Based Sequential Feature Aggregation for Intelligent Vehicles

IEEE TRANSACTIONS ON INTELLIGENT VEHICLES

引用

IEEE TRANSACTIONS ON INTELLIGENT VEHICLES 2024年第1期9卷 1512-1523页

作者： Jiang, Qi Hu, Chuan Zhao, Baixuan Huang, Yonghui Zhang, Xi Shanghai Jiao Tong Univ Sch Mech Engn Shanghai 200240 Peoples R China Shanghai Jiao Tong Univ Intelligent Vehicle Inst Sch Mech Engn Shanghai 200240 Peoples R China

3d object detection plays a key role in the perception system of intelligent vehicles. The reliable 3d structural information provided by LidAR points enables the accurate regression of position and pose, while the semantic ambiguity issue caused by the sparse points is still challenging. In this article, a scalable 3d object detection pipeline CenterSFA and a series of new modules are proposed to improve the detection performance. In contrast to previous point-level fusing models, semantic and geometric cues from images are sequentially utilized in a center-based paradigm. The object centers are accurately predicted with semantic guidance and selectively employed as the basis for feature aggregation and property regression. Specifically, the attention mechanism is utilized in the semantic and spatial similarity calculation, enabling the surrounding feature aggregation for multi-scale objects. An instance-level correlation is established between the camera feature and the BEV feature for cross-modal feature aggregation. Extensive experiments are conducted on the large-scale nuScenes dataset to verify the state-of-the-art performance of the proposed model, especially for occluded objects and far-range detection. The proposed model outperforms the competitive CenterPoint by 10.4% in mAP and 5.4% in NdS, as well as the representative fusion method MVP by 2.8% in mAP and 1.6% in NdS on val set, indicating its superiority in accurate 3d detection.

关键词： Three-dimensional displays Laser radar Feature extraction Proposals detectors Semantics object detection 3d object detection multi-sensor fusion intelligent vehicle perception feature aggregation

来源：评论

学校读者我要写书评

暂无评论

LXL: LidAR Excluded Lean 3d object detection With 4d Imaging Radar and Camera Fusion

IEEE TRANSACTIONS ON INTELLIGENT VEHICLES

引用

IEEE TRANSACTIONS ON INTELLIGENT VEHICLES 2024年第1期9卷 79-92页

作者： Xiong, Weiyi Liu, Jianan Huang, Tao Han, Qing-Long Xia, Yuxuan Zhu, Bing Beihang Univ Sch Automat Sci & Elect Engn Beijing 100191 Peoples R China Vitalent Consulting S-41761 Gothenburg Sweden James Cook Univ Coll Sci & Engn Cairns Qld 4878 Australia Swinburne Univ Technol Sch Sci Comp & Engn Technol Melbourne Vic 3122 Australia Chalmers Univ Technol Dept Elect Engn S-41296 Gothenburg Sweden

As an emerging technology and a relatively affordable device, the 4d imaging radar has already been confirmed effective in performing 3d object detection in autonomous driving. Nevertheless, the sparsity and noisiness of 4d radar point clouds hinder further performance improvement, and in-depth studies about its fusion with other modalities are lacking. On the other hand, as a new image view transformation strategy, "sampling" has been applied in a few image-based detectors and shown to outperform the widely applied "depth-based splatting" proposed in Lift-Splat-Shoot (LSS), even without image depth prediction. However, the potential of "sampling" is not fully unleashed. This article investigates the "sampling" view transformation strategy on the camera and 4dimaging radar fusion-based 3dobject detection. In the proposed LidAR Excluded Lean (LXL) model, predicted image depth distribution maps and radar 3d occupancy grids are generated from image perspective view (PV) features and radar bird's eye view (BEV) features, respectively. They are sent to the core of LXL, called "radar occupancy-assisted depth-based sampling", to aid image view transformation. We demonstrated that more accurate view transformation can be performed by introducing image depths and radar information to enhance the "sampling" strategy. Experiments on Vod and TJ4dRadSet datasets show that the proposed method outperforms the state-of-the-art 3d object detection methods by a significant margin without bells and whistles. Ablation studies demonstrate that our method performs the best among different enhancement settings.

关键词： Radar imaging Radar Three-dimensional displays Radar detection object detection Cameras Feature extraction 4d imaging radar camera multi-modal fusion 3d object detection deep learning autonomous driving

来源：评论

学校读者我要写书评

暂无评论

deployFusion: A deployable Monocular 3d object detection with Multi-Sensor Information Fusion in BEV for Edge devices

引用

SENSORS 2024年第21期24卷 7007页

作者： Huang, Fei Liu, Shengshu Zhang, Guangqian Hao, Bingsen Xiang, Yangkai Yuan, Kun China Rd & Bridge Corp Beijing 100010 Peoples R China Chongqing Seres Phoenix Intelligent Innovat Techno Chongqing 400039 Peoples R China Chongqing Jiaotong Univ Sch Mechatron & Vehicle Engn Chongqing 400074 Peoples R China

To address the challenges of suboptimal remote detection and significant computational burden in existing multi-sensor information fusion 3d object detection methods, a novel approach based on Bird's-Eye View (BEV) is proposed. This method utilizes an enhanced lightweight EdgeNeXt feature extraction network, incorporating residual branches to address network degradation caused by the excessive depth of STdA encoding blocks. Meantime, deformable convolution is used to expand the receptive field and reduce computational complexity. The feature fusion module constructs a two-stage fusion network to optimize the fusion and alignment of multi-sensor features. This network aligns image features to supplement environmental information with point cloud features, thereby obtaining the final BEV features. Additionally, a Transformer decoder that emphasizes global spatial cues is employed to process the BEV feature sequence, enabling precise detection of distant small objects. Experimental results demonstrate that this method surpasses the baseline network, with improvements of 4.5% in the NuScenes detection score and 5.5% in average precision for detection objects. Finally, the model is converted and accelerated using TensorRT tools for deployment on mobile devices, achieving an inference time of 138 ms per frame on the Jetson Orin NX embedded platform, thus enabling real-time 3d object detection.

关键词： multi-sensor information fusion 3d object detection BEV feature fusion model deployment

来源：评论

学校读者我要写书评

暂无评论

LidAR-Camera Fusion in Perspective View for 3d object detection in Surface Mine

IEEE TRANSACTIONS ON INTELLIGENT VEHICLES

引用

IEEE TRANSACTIONS ON INTELLIGENT VEHICLES 2024年第2期9卷 3721-3730页

作者： Ai, Yunfeng Yang, Xue Song, Ruiqi Cui, Chenglin Li, Xinqing Cheng, Qi Tian, Bin Chen, Long Univ Chinese Acad Sci Sch Artificial Intelligence Beijing 100049 Peoples R China Waytous Inc Qingdao 266109 Peoples R China Chinese Acad Sci Inst Automat State Key Lab Multimodal Artificial Intelligence S Beijing 100190 Peoples R China Tongji Univ Coll Surveying & Geoinformat Shanghai 200092 Peoples R China China Univ Min & Technol Sch Mech & Elect Engn Beijing 100083 Peoples R China China Univ Min & Technol Inner Mongolia Res Inst Ordos 017000 Peoples R China

LidAR-Camera fusion can effectively provide the complementary geometric and appearance information for 3d object detection task of autonomous driving system. However, current dominant methods designed for urban scenes often fuse multi-modal information into a Bird's-Eye-View (BEV) feature representation and have a relatively short perception range, which limits their application in surface mine. To achieve long-range perception in surface mine, we utilize a solid-state LidAR (i.e. Livox Horizon) that has a large perception range (over 200 meters). Accordingly, we propose PVFusion, a novel 3d object detection method that only fuses LidAR and camera information in Perspective View (PV). Specifically, we first generate the multi-channel depth map via point cloud projection and depth Map densification (dMd). Then, we extract visual-depth features and local geometry features from depth-enhanced camera image and point cloud respectively. To better fuse multi-modal information, we propose an Attentional Feature Fusion (AFF) module to dynamically fuse visual-depth features and local geometry features into fused PV features. Moreover, we design a three-branch detection head that leverages a depth decoupling Strategy (ddS) to improve the depth estimation for distant objects. We conduct extensive experiments on our surface mine dataset and verify the effectiveness of the proposed PVFusion.

关键词： Three-dimensional displays Feature extraction Laser radar Point cloud compression Cameras object detection Fuses 3d object detection autonomous driving LidAR-Camera fusion perspective view

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：