检索结果-内蒙古大学图书馆

deep Learning on 3d semantic segmentation: A detailed Review

REMOTE SENSING 2025年第2期17卷 298-298页

作者： Betsas, Thodoris Georgopoulos, Andreas doulamis, Anastasios Grussenmeyer, Pierre Natl Tech Univ Athens Sch Rural Surveying & Geoinformat Engn Lab Photogrammetry Athens 15772 Greece Univ Strasbourg ICube Lab INSA Strasbourg CNRSUMR 7357 F-67084 Strasbourg France

In this paper, an exhaustive review and comprehensive analysis of recent and former deep learning methods in 3d semantic segmentation (3dSS) is presented. In the related literature, the taxonomy scheme used for the classification of 3dSS deep learning methods is ambiguous. Based on the taxonomy schemes of nine existing review papers, a new taxonomy scheme for 3dSS deep learning methods is proposed, aiming to standardize it and improve the comparability and clarity across related studies. Furthermore, an extensive overview of the available 3dSS indoor and outdoor datasets is provided along with their links. The core part of this review is the detailed presentation of recent and former 3dSS deep learning methods and their classification using the proposed taxonomy scheme along with their GitHub repositories. Additionally, a brief but informative analysis of the evaluation metrics and loss functions used in 3dSS is included. Finally, a fruitful discussion of the examined 3dSS methods and datasets is presented to foster new research directions and applications in the field of 3dSS. In addition to this review, a GitHub repository is provided, including an initial classification of over 400 3dSS methods, using the proposed taxonomy scheme.

关键词： 3d semantic segmentation point clouds deep learning review

来源：评论

学校读者我要写书评

暂无评论

Robust 3d semantic segmentation Method Based on Multi-Modal Collaborative Learning

引用

REMOTE SENSING 2024年第3期16卷 453-453页

作者： Ni, Peizhou Li, Xu Xu, Wang Zhou, Xiaojing Jiang, Tao Hu, Weiming Southeast Univ Sch Instrument Sci & Engn Nanjing 210096 Peoples R China Xuzhou XCMG Automobile Mfg Co Ltd Xuzhou 221112 Peoples R China China Automot Engn Res Inst Co Ltd Chongqing 401122 Peoples R China Southeast Univ Sch Transportat Nanjing 211189 Peoples R China

Since camera and LidAR sensors provide complementary information for the 3d semantic segmentation of intelligent vehicles, extensive efforts have been invested to fuse information from multi-modal data. despite considerable advantages, fusion-based methods still have inevitable limitations: field-of-view disparity between two modal inputs, demanding precise paired data as inputs in both the training and inferring stages, and consuming more resources. These limitations pose significant obstacles to the practical application of fusion-based methods in real-world scenarios. Therefore, we propose a robust 3d semantic segmentation method based on multi-modal collaborative learning, aiming to enhance feature extraction and segmentation performance for point clouds. In practice, an attention based cross-modal knowledge distillation module is proposed to effectively acquire comprehensive information from multi-modal data and guide the pure point cloud network;then, a confidence-map-driven late fusion strategy is proposed to dynamically fuse the results of two modalities at the pixel-level to complement their advantages and further optimize segmentation results. The proposed method is evaluated on two public datasets (urban dataset semanticKITTI and off-road dataset RELLIS-3d) and our unstructured test set. The experimental results demonstrate the competitiveness of state-of-the-art methods in diverse scenarios and a robustness to sensor faults.

关键词： 3d semantic segmentation multi-modal fusion collaborative learning knowledge distillation

来源：评论

学校读者我要写书评

暂无评论

EPMF: Efficient Perception-Aware Multi-Sensor Fusion for 3d semantic segmentation

引用

IEEE TRANSACTIONS ON PATTERN ANALYSIS ANd MACHINE INTELLIGENCE 2024年第12期46卷 8258-8273页

作者： Tan, Mingkui Zhuang, Zhuangwei Chen, Sitao Li, Rong Jia, Kui Wang, Qicheng Li, Yuanqing South China Univ Technol Sch Software Engn Guangzhou 510641 Guangdong Peoples R China Pazhou Lab Guangzhou 510335 Peoples R China South China Univ Technol Sch Elect & Informat Engn Guangzhou 510641 Guangdong Peoples R China Hong Kong Univ Sci & Technol Dept Math Clear Water Bay Hong Kong Peoples R China Minieye Shenzhen 518063 Guangdong Peoples R China

We study multi-sensor fusion for 3d semantic segmentation that is important to scene understanding for many applications, such as autonomous driving and robotics. Existing fusion-based methods, however, may not achieve promising performance due to the vast difference between the two modalities. In this work, we investigate a collaborative fusion scheme called perception-aware multi-sensor fusion (PMF) to effectively exploit perceptual information from two modalities, namely, appearance information from RGB images and spatio-depth information from point clouds. To this end, we project point clouds to the camera coordinate using perspective projection, and process both inputs from LidAR and cameras in 2d space while preventing the information loss of RGB images. Then, we propose a two-stream network to extract features from the two modalities, separately. The extracted features are fused by effective residual-based fusion modules. Moreover, we introduce additional perception-aware losses to measure the perceptual difference between the two modalities. Last, we propose an improved version of PMF, i.e., EPMF, which is more efficient and effective by optimizing data pre-processing and network architecture under perspective projection. Specifically, we propose cross-modal alignment and cropping to obtain tight inputs and reduce unnecessary computational costs. We then explore more efficient contextual modules under perspective projection and fuse the LidAR features into the camera stream to boost the performance of the two-stream network. Extensive experiments on benchmark data sets show the superiority of our method. For example, on nuScenes test set, our EPMF outperforms the state-of-the-art method, i.e., RangeFormer, by 0.9% in mIoU.

关键词： Point cloud compression Laser radar Cameras semantic segmentation Three-dimensional displays Feature extraction Sensors 3d semantic segmentation autonomous driving deep neural networks multi-sensor fusion scene understanding

来源：评论

学校读者我要写书评

暂无评论

Robust 3d semantic segmentation Based on Multi-Phase Multi-Modal Fusion for Intelligent Vehicles

IEEE TRANSACTIONS ON INTELLIGENT VEHICLES

引用

IEEE TRANSACTIONS ON INTELLIGENT VEHICLES 2024年第1期9卷 1602-1614页

作者： Ni, Peizhou Li, Xu Xu, Wang Kong, dong Hu, Yue Wei, Kun Southeast Univ Sch Instrument Sci & Engn Nanjing 210096 Peoples R China

3d semantic segmentation is a key technology for intelligent vehicles. Recently, great efforts have been made to achieve accurate and robust 3d semantic segmentation results through LidAR-camera fusion, due to the complementary information of images like color and texture. However, current LidAR-camera fusion methods still lag behind LidAR-only methods in complex and varied scenarios containing sparse and similar geometric features. Three main factors limit their performance: sparse mappings between two modalities, the contribution of different modality features, and the gap between results of two modalities. Addressing that, we present a multi-phase LidAR-camera fusion method, which comprehensively fuses information of two modalities in three phases: first, a context sampler block is proposed to fuse low-level image contextual features into LidAR points;second, a cross dimensional fusion block is designed to implement the interaction of shared features for hierarchical supervision while retaining private features for jointly refining their predictions;third, a modality perceptual conformance loss is induced to make them learn from each other in an explicit manner by minimizing the gap between two modality predictions. Quantitative and qualitative evaluations with state-of-the-art (SOTA) methods are performed on urban dataset semanticKITTI, off-road dataset RELLIS-3d and our unstructured test set. The experimental results validate the competitiveness of our method in improving accuracy and scene-adaptivity.

关键词： Three-dimensional displays semantic segmentation Point cloud compression Laser radar Fuses Intelligent vehicles semantics Intelligent vehicles 3d semantic segmentation multi-modal fusion multi-phase

来源：评论

学校读者我要写书评

暂无评论

A Multi-Phase Camera-LidAR Fusion Network for 3d semantic segmentation With Weak Supervision

引用

IEEE TRANSACTIONS ON CIRCUITS ANd SYSTEMS FOR VIdEO TECHNOLOGY 2023年第8期33卷 3737-3746页

作者： Chang, Xuepeng Pan, Huihui Sun, Weichao Gao, Huijun Harbin Inst Technol Res Inst Intelligent Control & Syst Harbin 150001 Peoples R China Ningbo Inst Intelligent Equipment Technol Co Ltd Ningbo 315200 Peoples R China Yongjiang Lab Ningbo 315202 Peoples R China

Camera and LidAR are indispensable perception units in autonomous driving, providing complementary environmental information for 3d semantic segmentation. It is the key point that fuses the information of two modalities to accurate and robust semantic segmentation. However, three major factors will restrict the performance of fusion-based methods, i.e., the reliability of image features, the contribution of different image features, and the trade-off between results of image and point cloud. This paper proposes a novel multi-phase fusion network for 3d semantic segmentation. For the first factor, this paper takes the lead in regarding the problem that image features may be wrong due to the lack of dense annotations in the common datasets as a weak supervision problem and introduces the weakly supervised loss. Second, the proposed attention based feature fusion module can filter and reweight the image features effectively. Third, the results of the two modalities are further fused by self-confidence based late fusion module at pixel-level to complement their advantages. The proposed scheme has been evaluated on nuScenes and semanticKITTI benchmarks, and the results show the competitiveness with state-of-the-art methods. The ablation studies demonstrate the superiority of the method in sparse classes segmentation. In addition, the robustness is also evaluated, and the results of the proposed method can keep relatively accurate even when faults in one of the sensors.

关键词： Autonomous driving multi-modal fusion 3d semantic segmentation weak supervision

来源：评论

学校读者我要写书评

暂无评论

OA-CNNs: Omni-Adaptive Sparse CNNs for 3d semantic segmentation

OA-CNNs: Omni-Adaptive Sparse CNNs for 3D Semantic Segmentat...

引用

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Peng, Bohao Wu, Xiaoyang Jiang, Li Chen, Yukang Zhao, Hengshuang Tian, Zhuotao Jia, Jiaya CUHK Hong Kong Peoples R China HKU Hong Kong Peoples R China CUHK Shenzhen Peoples R China HIT Shenzhen Peoples R China

ISBN: (纸本)9798350353006

The booming of 3d recognition in the 2020s began with the introduction of point cloud transformers. They quickly overwhelmed sparse CNNs and became state-of-the-art models, especially in 3d semantic segmentation. However, sparse CNNs are still valuable networks, due to their efficiency treasure, and ease of application. In this work, we reexamine the design distinctions and test the limits of what a sparse CNN can achieve. We discover that the key credit to the performance difference is adaptivity. Specifically, we propose two key components, i.e., adaptive receptive fields (spatially) and adaptive relation, to bridge the gap. This exploration led to the creation of Omni-Adaptive 3d CNNs (OA-CNNs), a family of networks that integrates a lightweight module to greatly enhance the adaptivity of sparse CNNs at minimal computational cost. Without any self-attention modules, OA-CNNs favorably surpass point transformers in terms of accuracy in both indoor and outdoor scenes, with much less latency and memory cost. Notably, it achieves 76.1%, 78.9%, and 70.6% mIoU on ScanNet v2, nuScenes, and semanticKITTI validation benchmarks respectively, while maintaining at most 5x better speed than transformer counterparts. This revelation highlights the potential of pure sparse CNNs to outperform transformer-related networks. Our code is built upon Pointcept [9], which is available at here (1).

关键词： 3d semantic segmentation

来源：评论

学校读者我要写书评

暂无评论

Open-Vocabulary 3d semantic segmentation with Foundation Models

Open-Vocabulary 3D Semantic Segmentation with Foundation Mod...

引用

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Jiang, Li Shi, Shaoshuai Schiele, Bernt Max Planck Inst Informat Saarland Informat Campus Saarbrucken Germany

ISBN: (纸本)9798350353006

In dynamic 3d environments, the ability to recognize a diverse range of objects without the constraints of predefined categories is indispensable for real-world applications. In response to this need, we introduce OV3d, an innovative framework designed for open-vocabulary 3d semantic segmentation. OV3d leverages the broad open-world knowledge embedded in vision and language foundation models to establish a fine-grained correspondence between 3d points and textual entity descriptions. These entity descriptions are enriched with contextual information, enabling a more open and comprehensive understanding. By seamlessly aligning 3d point features with entity text features, OV3d empow-ers open-vocabulary recognition in the 3d domain, achieving state-of-the-art open-vocabulary semantic segmentation performance across multiple datasets, including ScanNet, Matterport3d, and nuScenes.

关键词： 3d semantic segmentation Foundation Models Open Vocabulary

来源：评论

学校读者我要写书评

暂无评论

CLFusion:3d semantic segmentation Based on Camera and Lidar Fusion

CLFusion:3D Semantic Segmentation Based on Camera and Lidar ...

引用

IEEE International Symposium on Circuits and Systems (ISCAS)

作者： Wang, Tianyue Song, Rujun Xiao, Zhuoling Yan, Bo Qin, Haojie He, di Univ Elect Sci & Technol China Sch Informat & Commun Engn Chengdu Peoples R China Shanghai Jiao Tong Univ Sch Elect Informat & Elect Engn Shanghai Peoples R China

ISBN: (纸本)9798350330991;9798350331004

In the field of autonomous driving, semantic segmentation is crucial for scene understanding. Currently, there are two main methods: camera-based and Lidar-based approaches. To address the issues of Lidar segmentation lacking texture features and image segmentation lacking distance information, this paper proposes a fusion of camera and Lidar to achieve 3d semantic segmentation. The method utilizes a dual-stream encoder-decoder network to process camera images and Lidar point cloud and incorporates a specially designed attention mechanism module for feature fusion. To avoid expensive manual annotation of 3d point clouds, the study also introduces a cross-dataset and cross-modal self-supervised training approach. Experimental results show a 2.4% improvement compared to the Lidar-only mode baseline results on the semanticKITTI dataset and a 6% improvement on the nuScenes dataset.

关键词： 3d semantic segmentation multi-modal fusion self-supervised training

来源：评论

学校读者我要写书评

暂无评论

ROBUST 3d semantic segmentation WITH INCOMPLETE POINT CLOUdS BASEd ON SEQUENTIAL FRAME SAMPLING 31

ROBUST 3D SEMANTIC SEGMENTATION WITH INCOMPLETE POINT CLOUDS...

引用

2024 International Conference on Image Processing

作者： Yamaguchi, Masahiro Higa, Kyota Hosoi, Toshinori Shibata, Takashi NEC Corp Ltd Visual Intelligence Res Labs Shimonumabe 1753Nakahara Ku Kawasaki Kanagawa Japan

ISBN: (纸本)9798350349405;9798350349399

This paper proposes a method for learning 3d semantic segmentation robust to incomplete point clouds. Our method first generates pseud-incomplete point clouds from original 3d point clouds by sequential frame sampling that creates multiple subsets considering the continuity of an RGB-d sequence for reproducing incomplete areas in the point clouds. It then simultaneously learns completion networks and semantic segmentation networks with the pseud-incomplete point clouds. We evaluate our method on the 3d semantic segmentation task. Experimental results on ScanNet v2, an indoor environment, show that our method improves mIoU by 0.4 points for the original point clouds and 6.3 points for the incomplete point clouds compared with a conventional method. Experimental results on WorkPlace dataset, an outdoor environment, show that our method improves mIoU by 6.5 points for the original point clouds and 11.1 points for the incomplete point clouds compared with the conventional method. These results improve the safety and operability of environmental awareness in applications such as robotics.

关键词： 3d semantic segmentation Point Completion Incomplete Point Clouds

来源：评论

学校读者我要写书评

暂无评论

Adaptive Margin Contrastive Learning for Ambiguity-aware 3d semantic segmentation

Adaptive Margin Contrastive Learning for Ambiguity-aware 3D ...

引用

IEEE International Conference on Multimedia and Expo (ICME)

作者： Chen, Yang duane, Yueqi Zhang, Runzhong Tan, Yap-Peng Nanyang Technol Univ Singapore Singapore Tsinghua Univ Beijing Peoples R China

ISBN: (纸本)9798350390155;9798350390162

In this paper, we propose an adaptive margin contrastive learning method for 3d point cloud semantic segmentation, namely AMContrast3d. Most existing methods use equally penalized objectives, which ignore per-point ambiguities and less discriminated features stemming from transition regions. However, as highly ambiguous points may be indistinguishable even for humans, their manually annotated labels are less reliable, and hard constraints over these points would lead to suboptimal models. To address this, we design adaptive objectives for individual points based on their ambiguity levels, aiming to ensure the correctness of low-ambiguity points while allowing mistakes for high-ambiguity points. Specifically, we first estimate ambiguities based on position embeddings. Then, we develop a margin generator to shift decision boundaries for contrastive feature embeddings, so margins are narrowed due to increasing ambiguities with even negative margins for extremely highambiguity points. Experimental results on large-scale datasets, S3dIS and ScanNet, demonstrate that our method outperforms state-of-the-art methods.

关键词： 3d semantic segmentation 3d Scene Understanding Contrastive Learning decision Boundary

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：