检索结果-内蒙古大学图书馆

deep Learning on 3d semantic segmentation: A detailed Review

REMOTE SENSING 2025年第2期17卷 298-298页

作者： Betsas, Thodoris Georgopoulos, Andreas doulamis, Anastasios Grussenmeyer, Pierre Natl Tech Univ Athens Sch Rural Surveying & Geoinformat Engn Lab Photogrammetry Athens 15772 Greece Univ Strasbourg ICube Lab INSA Strasbourg CNRSUMR 7357 F-67084 Strasbourg France

In this paper, an exhaustive review and comprehensive analysis of recent and former deep learning methods in 3d semantic segmentation (3dSS) is presented. In the related literature, the taxonomy scheme used for the classification of 3dSS deep learning methods is ambiguous. Based on the taxonomy schemes of nine existing review papers, a new taxonomy scheme for 3dSS deep learning methods is proposed, aiming to standardize it and improve the comparability and clarity across related studies. Furthermore, an extensive overview of the available 3dSS indoor and outdoor datasets is provided along with their links. The core part of this review is the detailed presentation of recent and former 3dSS deep learning methods and their classification using the proposed taxonomy scheme along with their GitHub repositories. Additionally, a brief but informative analysis of the evaluation metrics and loss functions used in 3dSS is included. Finally, a fruitful discussion of the examined 3dSS methods and datasets is presented to foster new research directions and applications in the field of 3dSS. In addition to this review, a GitHub repository is provided, including an initial classification of over 400 3dSS methods, using the proposed taxonomy scheme.

关键词： 3d semantic segmentation point clouds deep learning review

来源：评论

学校读者我要写书评

暂无评论

3d semantic segmentation Algorithm for Indoor Scenes based on Long-term Memory 15

3D Semantic Segmentation Algorithm for Indoor Scenes based o...

引用

15th IEEE Conference on Industrial Electronics and Applications (ICIEA)

作者： Liu, Ziyang Chen, Weihai Wang, Jianhua Wu, Xingming Yue, Haosong Peng, Zongju Li, Zhengguo Beihang Univ Sch Automat Sci & Elect Engn Beijing 100191 Peoples R China Ningbo Univ Fac Elect Engn & Comp Sci Ningbo 315211 Peoples R China Inst Infocomm Res Singapore 138632 Singapore

ISBN: (纸本)9781728151694

deep learning has a strong ability to tackle pixel-level labeling tasks in image understanding. However, the disorder and irregularity of 3d point cloud data make it difficult to be applied. Though there are a few approaches that have acquired satisfied results on 3d data processing recently, there is no consideration of the relationships between each point, which plays an important role in indoor objects semantic segmentation and has been proved on plane images. In this paper, a spatial long-term connection of points is modeled by a fully connected conditional random field (CRF) to enhance the memory ability of the network. Since it is infeasible to compute due to the dimension explosion caused by potentially fully connected CRF, a highly efficient method in utilizing linear combination of multiple Gaussian kernels is proposed to approximate it. Finally, the CM;has been integrated into the deep learning framework, making it trainable end-to-end. The test evaluation of the experiments show that this network reaches current benchmarks and have a strong performance on public 3d indoor datasets. From these results, it is concluded that the relationships of each pair of points are conductive to semantic segmentation for indoor scenes.

关键词： 3d semantic segmentation deep learning conditional random field long-term memory

来源：评论

学校读者我要写书评

暂无评论

Robust 3d semantic segmentation Method Based on Multi-Modal Collaborative Learning

引用

REMOTE SENSING 2024年第3期16卷 453-453页

作者： Ni, Peizhou Li, Xu Xu, Wang Zhou, Xiaojing Jiang, Tao Hu, Weiming Southeast Univ Sch Instrument Sci & Engn Nanjing 210096 Peoples R China Xuzhou XCMG Automobile Mfg Co Ltd Xuzhou 221112 Peoples R China China Automot Engn Res Inst Co Ltd Chongqing 401122 Peoples R China Southeast Univ Sch Transportat Nanjing 211189 Peoples R China

Since camera and LidAR sensors provide complementary information for the 3d semantic segmentation of intelligent vehicles, extensive efforts have been invested to fuse information from multi-modal data. despite considerable advantages, fusion-based methods still have inevitable limitations: field-of-view disparity between two modal inputs, demanding precise paired data as inputs in both the training and inferring stages, and consuming more resources. These limitations pose significant obstacles to the practical application of fusion-based methods in real-world scenarios. Therefore, we propose a robust 3d semantic segmentation method based on multi-modal collaborative learning, aiming to enhance feature extraction and segmentation performance for point clouds. In practice, an attention based cross-modal knowledge distillation module is proposed to effectively acquire comprehensive information from multi-modal data and guide the pure point cloud network;then, a confidence-map-driven late fusion strategy is proposed to dynamically fuse the results of two modalities at the pixel-level to complement their advantages and further optimize segmentation results. The proposed method is evaluated on two public datasets (urban dataset semanticKITTI and off-road dataset RELLIS-3d) and our unstructured test set. The experimental results demonstrate the competitiveness of state-of-the-art methods in diverse scenarios and a robustness to sensor faults.

关键词： 3d semantic segmentation multi-modal fusion collaborative learning knowledge distillation

来源：评论

学校读者我要写书评

暂无评论

EPMF: Efficient Perception-Aware Multi-Sensor Fusion for 3d semantic segmentation

引用

IEEE TRANSACTIONS ON PATTERN ANALYSIS ANd MACHINE INTELLIGENCE 2024年第12期46卷 8258-8273页

作者： Tan, Mingkui Zhuang, Zhuangwei Chen, Sitao Li, Rong Jia, Kui Wang, Qicheng Li, Yuanqing South China Univ Technol Sch Software Engn Guangzhou 510641 Guangdong Peoples R China Pazhou Lab Guangzhou 510335 Peoples R China South China Univ Technol Sch Elect & Informat Engn Guangzhou 510641 Guangdong Peoples R China Hong Kong Univ Sci & Technol Dept Math Clear Water Bay Hong Kong Peoples R China Minieye Shenzhen 518063 Guangdong Peoples R China

We study multi-sensor fusion for 3d semantic segmentation that is important to scene understanding for many applications, such as autonomous driving and robotics. Existing fusion-based methods, however, may not achieve promising performance due to the vast difference between the two modalities. In this work, we investigate a collaborative fusion scheme called perception-aware multi-sensor fusion (PMF) to effectively exploit perceptual information from two modalities, namely, appearance information from RGB images and spatio-depth information from point clouds. To this end, we project point clouds to the camera coordinate using perspective projection, and process both inputs from LidAR and cameras in 2d space while preventing the information loss of RGB images. Then, we propose a two-stream network to extract features from the two modalities, separately. The extracted features are fused by effective residual-based fusion modules. Moreover, we introduce additional perception-aware losses to measure the perceptual difference between the two modalities. Last, we propose an improved version of PMF, i.e., EPMF, which is more efficient and effective by optimizing data pre-processing and network architecture under perspective projection. Specifically, we propose cross-modal alignment and cropping to obtain tight inputs and reduce unnecessary computational costs. We then explore more efficient contextual modules under perspective projection and fuse the LidAR features into the camera stream to boost the performance of the two-stream network. Extensive experiments on benchmark data sets show the superiority of our method. For example, on nuScenes test set, our EPMF outperforms the state-of-the-art method, i.e., RangeFormer, by 0.9% in mIoU.

关键词： Point cloud compression Laser radar Cameras semantic segmentation Three-dimensional displays Feature extraction Sensors 3d semantic segmentation autonomous driving deep neural networks multi-sensor fusion scene understanding

来源：评论

学校读者我要写书评

暂无评论

Robust 3d semantic segmentation Based on Multi-Phase Multi-Modal Fusion for Intelligent Vehicles

IEEE TRANSACTIONS ON INTELLIGENT VEHICLES

引用

IEEE TRANSACTIONS ON INTELLIGENT VEHICLES 2024年第1期9卷 1602-1614页

作者： Ni, Peizhou Li, Xu Xu, Wang Kong, dong Hu, Yue Wei, Kun Southeast Univ Sch Instrument Sci & Engn Nanjing 210096 Peoples R China

3d semantic segmentation is a key technology for intelligent vehicles. Recently, great efforts have been made to achieve accurate and robust 3d semantic segmentation results through LidAR-camera fusion, due to the complementary information of images like color and texture. However, current LidAR-camera fusion methods still lag behind LidAR-only methods in complex and varied scenarios containing sparse and similar geometric features. Three main factors limit their performance: sparse mappings between two modalities, the contribution of different modality features, and the gap between results of two modalities. Addressing that, we present a multi-phase LidAR-camera fusion method, which comprehensively fuses information of two modalities in three phases: first, a context sampler block is proposed to fuse low-level image contextual features into LidAR points;second, a cross dimensional fusion block is designed to implement the interaction of shared features for hierarchical supervision while retaining private features for jointly refining their predictions;third, a modality perceptual conformance loss is induced to make them learn from each other in an explicit manner by minimizing the gap between two modality predictions. Quantitative and qualitative evaluations with state-of-the-art (SOTA) methods are performed on urban dataset semanticKITTI, off-road dataset RELLIS-3d and our unstructured test set. The experimental results validate the competitiveness of our method in improving accuracy and scene-adaptivity.

关键词： Three-dimensional displays semantic segmentation Point cloud compression Laser radar Fuses Intelligent vehicles semantics Intelligent vehicles 3d semantic segmentation multi-modal fusion multi-phase

来源：评论

学校读者我要写书评

暂无评论

A Multi-Phase Camera-LidAR Fusion Network for 3d semantic segmentation With Weak Supervision

引用

IEEE TRANSACTIONS ON CIRCUITS ANd SYSTEMS FOR VIdEO TECHNOLOGY 2023年第8期33卷 3737-3746页

作者： Chang, Xuepeng Pan, Huihui Sun, Weichao Gao, Huijun Harbin Inst Technol Res Inst Intelligent Control & Syst Harbin 150001 Peoples R China Ningbo Inst Intelligent Equipment Technol Co Ltd Ningbo 315200 Peoples R China Yongjiang Lab Ningbo 315202 Peoples R China

Camera and LidAR are indispensable perception units in autonomous driving, providing complementary environmental information for 3d semantic segmentation. It is the key point that fuses the information of two modalities to accurate and robust semantic segmentation. However, three major factors will restrict the performance of fusion-based methods, i.e., the reliability of image features, the contribution of different image features, and the trade-off between results of image and point cloud. This paper proposes a novel multi-phase fusion network for 3d semantic segmentation. For the first factor, this paper takes the lead in regarding the problem that image features may be wrong due to the lack of dense annotations in the common datasets as a weak supervision problem and introduces the weakly supervised loss. Second, the proposed attention based feature fusion module can filter and reweight the image features effectively. Third, the results of the two modalities are further fused by self-confidence based late fusion module at pixel-level to complement their advantages. The proposed scheme has been evaluated on nuScenes and semanticKITTI benchmarks, and the results show the competitiveness with state-of-the-art methods. The ablation studies demonstrate the superiority of the method in sparse classes segmentation. In addition, the robustness is also evaluated, and the results of the proposed method can keep relatively accurate even when faults in one of the sensors.

关键词： Autonomous driving multi-modal fusion 3d semantic segmentation weak supervision

来源：评论

学校读者我要写书评

暂无评论

Feature fusion network based on attention mechanism for 3d semantic segmentation of point clouds

引用

PATTERN RECOGNITION LETTERS 2020年 133卷 327-333页

作者： Zhou, Heng Fang, Zhijun Gao, Yongbin Huang, Bo Zhong, Cengsi Shang, Ruoxi Shanghai Univ Engn Sci 333 Longteng Rd Shanghai 201620 Peoples R China Univ Calif Berkeley Berkeley CA 94720 USA

3d scene parsing has always been a hot topic and point clouds are efficient data format to represent scenes. The semantic segmentation of point clouds is critical to the 3d scene, which is a challenging problem due to the unordered structure of point clouds. The max-pooling operation is typically used to obtain the order invariant features, while the point-wise features are destroyed after the max-pooling operation. In this paper, we propose a feature fusion network that fuses point-wise features and local features by attention mechanism to compensate for the loss caused by max-pooling operation. By incorporating point-wise features into local features, the point-wise variation is preserved to obtain a refined segmentation accuracy, and the attention mechanism is used to measure the importance of the pointwise features and local features for each 3d point. Extensive experiments show that our method achieves better performances than other prestigious methods. (C) 2020 Published by Elsevier B.V.

关键词： 3d semantic segmentation Point clouds Feature fusion Attention mechanism

来源：评论

学校读者我要写书评

暂无评论

decoupled Iterative deep Sensor Fusion for 3d semantic segmentation

引用

INTERNATIONAL JOURNAL OF semantic COMPUTING 2021年第3期15卷 293-312页

作者： duerr, Fabian Weigel, Hendrik Beyerer, Juergen AUDI AG D-85057 Ingolstadt Germany Karlsruhe Inst Technol KIT Vis & Fus Lab Karlsruhe Germany Fraunhofer Ctr Machine Learning Fraunhofer Inst Optron Syst Technol & Image Explo D-76131 Karlsruhe Germany

One of the key tasks for autonomous vehicles or robots is a robust perception of their 3d environment, which is why autonomous vehicles or robots are equipped with a wide range of different sensors. Building upon a robust sensor setup, understanding and interpreting their 3d environment is the next important step. semantic segmentation of 3d sensor data, e.g. point clouds, provides valuable information for this task and is often seen as key enabler for 3d scene understanding. This work presents an iterative deep fusion architecture for semantic segmentation of 3d point clouds, which builds upon a range image representation of the point clouds and additionally exploits camera features to increase accuracy and robustness. In contrast to other approaches, which fuse lidar and camera features once, the proposed fusion strategy iteratively combines and refines lidar and camera features at different scales inside the network architecture. Additionally, the proposed approach can deal with camera failure as well as jointly predict lidar and camera segmentation. We demonstrate the benefits of the presented iterative deep fusion approach on two challenging datasets, outperforming all range image-based lidar and fusion approaches. An in-depth evaluation underlines the effectiveness of the proposed fusion strategy and the potential of camera features for 3d semantic segmentation.

关键词： 3d semantic segmentation 2d semantic segmentation deep fusion convolutional neural networks point clouds sensor fusion

来源：评论

学校读者我要写书评

暂无评论

denseFuseNet: Improve 3d semantic segmentation in the Context of Autonomous driving with dense Correspondence

DenseFuseNet: Improve 3D Semantic Segmentation in the Contex...

引用

IEEE International Conference on Consumer Electronics and Computer Engineering (ICCECE)

作者： Lu, Yulun Shanghai Jianping High Sch Shanghai Peoples R China

ISBN: (纸本)9781728183190

With the development of deep convolutional networks, autonomous driving has been reforming human social activities in the recent decade. The core issue of autonomous driving is how to integrate the multi-modal perception system effectively, that is, using sensors such as lidar, RGB camera, and radar to identify general objects in traffic scenes. Extensive investigation shows that lidar and cameras are the two most powerful sensors widely used by autonomous driving companies such as Tesla and Waymo, which indeed revealed that how to integrate them effectively is bound to be one of the core issues in the field of autonomous driving in the future. Obviously, these two kinds of sensors have their inherent advantages and disadvantages. Based on the previous research works, we are motivated to fuse lidars and RGB cameras together to build a more robust perception system. It is not easy to design a model with two different domains from scratch, and a large number of previous works (e.g., FuseSeg [10]) has sufficiently proved that merging the RGB camera and lidar models can attain better results on vision tasks than the lidar model alone. However, it cannot adequately handle the inherent correspondence between the RGB camera and lidar data but rather arbitrarily interpolates between them, which quickly leads to severe distortion, heavy computational burden, and diminishing performance. To address these problems, in this paper, we proposed a general framework to establish a connection between lidar and RGB camera sensors, matching and fusing the features of the lidar and RGB models. We also defined two kinds of inaccuracies (missing pixels and covered points) in spherical projection and conducted a numerical analysis on them. Furthermore, we proposed an efficient filling algorithm to remedy the impact of missing pixels. Finally, we proposed a 3d semantic segmentation model, denseFuseNet, which incorporated our techniques and achieved a noticeable 5.8 and 14.2 improvement in

关键词： Autonomous driving 3d semantic segmentation lidar point clouds sensor fusion spherical projection noise analysis convolutional neural networks

来源：评论

学校读者我要写书评

暂无评论

OA-CNNs: Omni-Adaptive Sparse CNNs for 3d semantic segmentation

OA-CNNs: Omni-Adaptive Sparse CNNs for 3D Semantic Segmentat...

引用

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Peng, Bohao Wu, Xiaoyang Jiang, Li Chen, Yukang Zhao, Hengshuang Tian, Zhuotao Jia, Jiaya CUHK Hong Kong Peoples R China HKU Hong Kong Peoples R China CUHK Shenzhen Peoples R China HIT Shenzhen Peoples R China

ISBN: (纸本)9798350353006

The booming of 3d recognition in the 2020s began with the introduction of point cloud transformers. They quickly overwhelmed sparse CNNs and became state-of-the-art models, especially in 3d semantic segmentation. However, sparse CNNs are still valuable networks, due to their efficiency treasure, and ease of application. In this work, we reexamine the design distinctions and test the limits of what a sparse CNN can achieve. We discover that the key credit to the performance difference is adaptivity. Specifically, we propose two key components, i.e., adaptive receptive fields (spatially) and adaptive relation, to bridge the gap. This exploration led to the creation of Omni-Adaptive 3d CNNs (OA-CNNs), a family of networks that integrates a lightweight module to greatly enhance the adaptivity of sparse CNNs at minimal computational cost. Without any self-attention modules, OA-CNNs favorably surpass point transformers in terms of accuracy in both indoor and outdoor scenes, with much less latency and memory cost. Notably, it achieves 76.1%, 78.9%, and 70.6% mIoU on ScanNet v2, nuScenes, and semanticKITTI validation benchmarks respectively, while maintaining at most 5x better speed than transformer counterparts. This revelation highlights the potential of pure sparse CNNs to outperform transformer-related networks. Our code is built upon Pointcept [9], which is available at here (1).

关键词： 3d semantic segmentation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：