检索结果-内蒙古大学图书馆

31st ACM International Conference on Multimedia (MM)

作者： Wu, Yao Xing, Mingwei Zhang, Yachao Xie, Yuan Fan, Jianping Shi, Zhongchao Qu, Yanyun Xiamen Univ Sch Informat Xiamen Peoples R China Xiamen Univ Inst Artificial Intelligence Xiamen Peoples R China Tsinghua Univ Shenzhen Peoples R China East China Normal Univ Shanghai Peoples R China East China Normal Univ Chongqing Inst Chongqing Peoples R China Lenovo Res Beijing Peoples R China

ISBN: (纸本)9798400701085

Cross-modal Unsupervised domain Adaptation (UdA) becomes a research hotspot because it reduces the laborious annotation of target domain samples. Existing methods only mutually mimic the outputs of cross-modality in each domain, which enforces the class probability distribution agreeable in different domains. However, these methods ignore the complementarity brought by the modality fusion representation in cross-modal learning. In this paper, we propose a cross-modal UdA method for 3d semantic segmentation via Bidirectional Fusion-then-distillation, named BFtd-xMUdA, which explores cross-modal fusion in UdA and realizes distribution consistency between outputs of two domains not only for 2d image and 3d point cloud but also for 2d/3d and fusion. Our method contains three significant components: Model-agnostic Feature Fusion Module (MFFM), Bidirectional distillation (B-distill), and Cross-modal debiased Pseudo-Labeling (xdPL). MFFM is employed to generate cross-modal fusion features for establishing a latent space, which enforces maximum correlation and complementarity between two heterogeneous modalities. B-distill is introduced to exploit bidirectional knowledge distillation which includes cross-modality and cross-domain fusion distillation, and well-achieving domain-modality alignment. xdPL is designed to model the uncertainty of pseudo-labels by self-training scheme. Extensive experimental results demonstrate that our method outperforms state-of-the-art competitors in several adaptation scenarios.

关键词： 3d semantic segmentation Unsupervised domain adaptation

来源：评论

学校读者我要写书评

暂无评论

A Multi-Modal Fusion 3d semantic segmentation Method 3

A Multi-Modal Fusion 3D Semantic Segmentation Method

引用

3rd International Conference on Electronic Information Engineering and Computer Science, EIECS 2023

作者： Bi, Yuxuan Liu, Peng Shi, Jialin Zhang, Tianyi Changchun University of Science and Technology School of Electronic Information Engineering Changchun China

ISBN: (纸本)9798350316674

In order to address the issue of low accuracy in existing multi-modal fusion 3d semantic segmentation models, this study proposes an improved multimodal fusion 3d semantic segmentation model based on LidAR and camera. The model designs a Virtual Point Generation algorithm based on Multi-View Projection, which generates virtual points with depth and semantic information in 3d space by utilizing 2d image space, thereby enhancing the model's capability of extracting high-level features. Additionally, the model extends its receptive field to the 2d image space during the virtual point generation process, addressing the problem of virtual point noise that is difficult to eliminate in 3d space. Compared to other mainstream 3d semantic segmentation models, the proposed segmentation model in this paper achieves an accuracy improvement of 1.29% and 0.18% on the Waymo and nuScenes datasets, respectively. Experimental results demonstrate a significant enhancement in accuracy for the multimodal fusion 3d semantic segmentation model proposed in this study. © 2023 IEEE.

关键词： 3d semantic segmentation multi-modal fusion Multi-View Projection

来源：评论

学校读者我要写书评

暂无评论

Language-Grounded Indoor 3d semantic segmentation in the Wild 1

引用

17th European Conference on Computer Vision (ECCV)

作者： Rozenberszki, david Litany, Or dai, Angela Tech Univ Munich Munich Germany NVIDIA Santa Clara CA USA

ISBN: (数字)9783031198274

ISBN: (纸本)9783031198267;9783031198274

Recent advances in 3d semantic segmentation with deep neural networks have shown remarkable success, with rapid performance increase on available datasets. However, current 3d semantic segmentation benchmarks contain only a small number of categories - less than 30 for ScanNet and semanticKITTI, for instance, which are not enough to reflect the diversity of real environments (e.g., semantic image understanding covers hundreds to thousands of classes). Thus, we propose to study a larger vocabulary for 3d semantic segmentation with a new extended benchmark on ScanNet data with 200 class categories, an order of magnitude more than previously studied. This large number of class categories also induces a large natural class imbalance, both of which are challenging for existing 3d semantic segmentation methods. To learn more robust 3d features in this context, we propose a language-driven pre-training method to encourage learned 3d features that might have limited training examples to lie close to their pre-trained text embeddings. Extensive experiments show that our approach consistently outperforms state-of-the-art 3d pre-training for 3d semantic segmentation on our proposed benchmark (+9% relative mIoU), including limited-data scenarios with +25% relative mIoU using only 5% annotations.

关键词： 3d semantic scene understanding 3d semantic segmentation 3d representation learning Language+3d vision

来源：评论

学校读者我要写书评

暂无评论

dOdA: data-Oriented Sim-to-Real domain Adaptation for 3d semantic segmentation 1

引用

17th European Conference on Computer Vision (ECCV)

作者： ding, Runyu Yang, Jihan Jiang, Li Qi, Xiaojuan Univ Hong Kong Hong Kong Peoples R China MPI Informat Saarbrucken Germany

ISBN: (数字)9783031198120

ISBN: (纸本)9783031198113;9783031198120

deep learning approaches achieve prominent success in 3d semantic segmentation. However, collecting densely annotated real-world 3d datasets is extremely time-consuming and expensive. Training models on synthetic data and generalizing on real-world scenarios becomes an appealing alternative, but unfortunately suffers from notorious domain shifts. In this work, we propose a data-Oriented domain Adaptation (dOdA) framework to mitigate pattern and context gaps caused by different sensing mechanisms and layout placements across domains. Our dOdA encompasses virtual scan simulation to imitate real-world point cloud patterns and tail-aware cuboid mixing to alleviate the interior context gap with a cuboid-based intermediate domain. The first unsupervised sim-to-real adaptation benchmark on 3d indoor semantic segmentation is also built on 3d-FRONT, ScanNet and S3dIS along with 8 popular Unsupervised domain Adaptation (UdA) methods. Our dOdA surpasses existing UdA approaches by over 13% on both 3d-FRONT -> ScanNet and 3d-FRONT -> S3dIS. Code is available at https://***/CVMI-Lab/dOdA.

关键词： domain adaptation 3d semantic segmentation

来源：评论

学校读者我要写书评

暂无评论

Cross-domain and Cross-Modal Knowledge distillation in domain Adaptation for 3d semantic segmentation 22

Cross-Domain and Cross-Modal Knowledge Distillation in Domai...

引用

30th ACM International Conference on Multimedia (MM)

作者： Li, Miaoyu Zhang, Yachao Xie, Yuan Gao, Zuodong Li, Cuihua Zhang, Zhizhong Qu, Yanyun Xiamen Univ Xiamen Peoples R China East China Normal Univ Shanghai Peoples R China

ISBN: (纸本)9781450392037

With the emergence of multi-modal datasets where LidAR and camera are synchronized and calibrated, cross-modal Unsupervised domain Adaptation (UdA) has attracted increasing attention because it reduces the laborious annotation of target domain samples. To alleviate the distribution gap between source and target domains, existing methods conduct feature alignment by using adversarial learning. However, it is well-known to be highly sensitive to hyperparameters and difficult to train. In this paper, we propose a novel model (dual-Cross) that integrates Cross-domain Knowledge distillation (CdKd) and Cross-Modal Knowledge distillation (CMKd) to mitigate domain shift. Specifically, we design the multi-modal style transfer to convert source image and point cloud to target style. With these synthetic samples as input, we introduce a target-aware teacher network to learn knowledge of the target domain. Then we present dual-cross knowledge distillation when the student is learning on source domain. CdKd constrains teacher and student predictions under same modality to be consistent. It can transfer target-aware knowledge from the teacher to the student, making the student more adaptive to the target domain. CMKd generates hybrid-modal prediction from the teacher predictions and constrains it to be consistent with both 2d and 3d student predictions. It promotes the information interaction between two modalities to make them complement each other. From the evaluation results on various domain adaptation settings, dual-Cross significantly outperforms both uni-modal and cross-modal state-of-the-art methods.

关键词： Unsupervised domain adaptation Target-aware teacher network Cross-domain knowledge distillation 3d semantic segmentation Multi-modal style transfer

来源：评论

学校读者我要写书评

暂无评论

decoupled Iterative deep Sensor Fusion for 3d semantic segmentation

引用

INTERNATIONAL JOURNAL OF semantic COMPUTING 2021年第3期15卷 293-312页

作者： duerr, Fabian Weigel, Hendrik Beyerer, Juergen AUDI AG D-85057 Ingolstadt Germany Karlsruhe Inst Technol KIT Vis & Fus Lab Karlsruhe Germany Fraunhofer Ctr Machine Learning Fraunhofer Inst Optron Syst Technol & Image Explo D-76131 Karlsruhe Germany

One of the key tasks for autonomous vehicles or robots is a robust perception of their 3d environment, which is why autonomous vehicles or robots are equipped with a wide range of different sensors. Building upon a robust sensor setup, understanding and interpreting their 3d environment is the next important step. semantic segmentation of 3d sensor data, e.g. point clouds, provides valuable information for this task and is often seen as key enabler for 3d scene understanding. This work presents an iterative deep fusion architecture for semantic segmentation of 3d point clouds, which builds upon a range image representation of the point clouds and additionally exploits camera features to increase accuracy and robustness. In contrast to other approaches, which fuse lidar and camera features once, the proposed fusion strategy iteratively combines and refines lidar and camera features at different scales inside the network architecture. Additionally, the proposed approach can deal with camera failure as well as jointly predict lidar and camera segmentation. We demonstrate the benefits of the presented iterative deep fusion approach on two challenging datasets, outperforming all range image-based lidar and fusion approaches. An in-depth evaluation underlines the effectiveness of the proposed fusion strategy and the potential of camera features for 3d semantic segmentation.

关键词： 3d semantic segmentation 2d semantic segmentation deep fusion convolutional neural networks point clouds sensor fusion

来源：评论

学校读者我要写书评

暂无评论

From CAd Models to Soft Point Cloud Labels: An Automatic Annotation Pipeline for Cheaply Supervised 3d semantic segmentation

引用

REMOTE SENSING 2023年第14期15卷 3578-3578页

作者： Humblot-Renaux, Galadrielle Jensen, Simon Buus Mogelmose, Andreas Aalborg Univ Visual Anal & Percept Lab DK-9000 Aalborg Denmark

We propose a fully automatic annotation scheme that takes a raw 3d point cloud with a set of fitted CAd models as input and outputs convincing point-wise labels that can be used as cheap training data for point cloud segmentation. Compared with manual annotations, we show that our automatic labels are accurate while drastically reducing the annotation time and eliminating the need for manual intervention or dataset-specific parameters. Our labeling pipeline outputs semantic classes and soft point-wise object scores, which can either be binarized into standard one-hot-encoded labels, thresholded into weak labels with ambiguous points left unlabeled, or used directly as soft labels during training. We evaluate the label quality and segmentation performance of PointNet++ on a dataset of real industrial point clouds and Scan2CAd, a public dataset of indoor scenes. Our results indicate that reducing supervision in areas that are more difficult to label automatically is beneficial compared with the conventional approach of naively assigning a hard "best guess" label to every point.

关键词： 3d semantic segmentation automatic labeling soft labels point clouds deep learning

来源：评论

学校读者我要写书评

暂无评论

Learning Virtual View Selection for 3d Scene semantic segmentation

引用

IEEE TRANSACTIONS ON IMAGE PROCESSING 2024年 33卷 4159-4172页

作者： Mu, Tai-Jiang Shen, Ming-Yuan Lai, Yu-Kun Hu, Shi-Min Tsinghua Univ Minist Educ Key Lab Pervas Comp Beijing 100084 Peoples R China Tsinghua Univ Dept Comp Sci & Technol Beijing 100084 Peoples R China Cardiff Univ Sch Comp Sci & Informat Cardiff CF24 4AG Wales

2d-3d joint learning is essential and effective for fundamental 3d vision tasks, such as 3d semantic segmentation, due to the complementary information these two visual modalities contain. Most current 3d scene semantic segmentation methods process 2d images "as they are", i.e., only real captured 2d images are used. However, such captured 2d images may be redundant, with abundant occlusion and/or limited field of view (FoV), leading to poor performance for the current methods involving 2d inputs. In this paper, we propose a general learning framework for joint 2d-3d scene understanding by selecting informative virtual 2d views of the underlying 3d scene. We then feed both the 3d geometry and the generated virtual 2d views into any joint 2d-3d-input or pure 3d-input based deep neural models for improving 3d scene understanding. Specifically, we generate virtual 2d views based on an information score map learned from the current 3d scene semantic segmentation results. To achieve this, we formalize the learning of the information score map as a deep reinforcement learning process, which rewards good predictions using a deep neural network. To obtain a compact set of virtual 2d views that jointly cover informative surfaces of the 3d scene as much as possible, we further propose an efficient greedy virtual view coverage strategy in the normal-sensitive 6d space, including 3-dimensional point coordinates and 3-dimensional normal. We have validated our proposed framework for various joint 2d-3d-input or pure 3d-input based deep neural models on two real-world 3d scene datasets, i.e., ScanNet v2 and S3dIS, and the results demonstrate that our method obtains a consistent gain over baseline models and achieves new top accuracy for joint 2d and 3d scene semantic segmentation.

关键词： Three-dimensional displays semantic segmentation Solid modeling semantics Geometry Task analysis deep reinforcement learning Virtual view selection 2d-3d joint learning deep reinforcement learning 3d semantic segmentation

来源：评论

学校读者我要写书评

暂无评论

3d semantic segmentation Algorithm for Indoor Scenes based on Long-term Memory 15

3D Semantic Segmentation Algorithm for Indoor Scenes based o...

引用

15th IEEE Conference on Industrial Electronics and Applications (ICIEA)

作者： Liu, Ziyang Chen, Weihai Wang, Jianhua Wu, Xingming Yue, Haosong Peng, Zongju Li, Zhengguo Beihang Univ Sch Automat Sci & Elect Engn Beijing 100191 Peoples R China Ningbo Univ Fac Elect Engn & Comp Sci Ningbo 315211 Peoples R China Inst Infocomm Res Singapore 138632 Singapore

ISBN: (纸本)9781728151694

deep learning has a strong ability to tackle pixel-level labeling tasks in image understanding. However, the disorder and irregularity of 3d point cloud data make it difficult to be applied. Though there are a few approaches that have acquired satisfied results on 3d data processing recently, there is no consideration of the relationships between each point, which plays an important role in indoor objects semantic segmentation and has been proved on plane images. In this paper, a spatial long-term connection of points is modeled by a fully connected conditional random field (CRF) to enhance the memory ability of the network. Since it is infeasible to compute due to the dimension explosion caused by potentially fully connected CRF, a highly efficient method in utilizing linear combination of multiple Gaussian kernels is proposed to approximate it. Finally, the CM;has been integrated into the deep learning framework, making it trainable end-to-end. The test evaluation of the experiments show that this network reaches current benchmarks and have a strong performance on public 3d indoor datasets. From these results, it is concluded that the relationships of each pair of points are conductive to semantic segmentation for indoor scenes.

关键词： 3d semantic segmentation deep learning conditional random field long-term memory

来源：评论

学校读者我要写书评

暂无评论

denseFuseNet: Improve 3d semantic segmentation in the Context of Autonomous driving with dense Correspondence

DenseFuseNet: Improve 3D Semantic Segmentation in the Contex...

引用

IEEE International Conference on Consumer Electronics and Computer Engineering (ICCECE)

作者： Lu, Yulun Shanghai Jianping High Sch Shanghai Peoples R China

ISBN: (纸本)9781728183190

With the development of deep convolutional networks, autonomous driving has been reforming human social activities in the recent decade. The core issue of autonomous driving is how to integrate the multi-modal perception system effectively, that is, using sensors such as lidar, RGB camera, and radar to identify general objects in traffic scenes. Extensive investigation shows that lidar and cameras are the two most powerful sensors widely used by autonomous driving companies such as Tesla and Waymo, which indeed revealed that how to integrate them effectively is bound to be one of the core issues in the field of autonomous driving in the future. Obviously, these two kinds of sensors have their inherent advantages and disadvantages. Based on the previous research works, we are motivated to fuse lidars and RGB cameras together to build a more robust perception system. It is not easy to design a model with two different domains from scratch, and a large number of previous works (e.g., FuseSeg [10]) has sufficiently proved that merging the RGB camera and lidar models can attain better results on vision tasks than the lidar model alone. However, it cannot adequately handle the inherent correspondence between the RGB camera and lidar data but rather arbitrarily interpolates between them, which quickly leads to severe distortion, heavy computational burden, and diminishing performance. To address these problems, in this paper, we proposed a general framework to establish a connection between lidar and RGB camera sensors, matching and fusing the features of the lidar and RGB models. We also defined two kinds of inaccuracies (missing pixels and covered points) in spherical projection and conducted a numerical analysis on them. Furthermore, we proposed an efficient filling algorithm to remedy the impact of missing pixels. Finally, we proposed a 3d semantic segmentation model, denseFuseNet, which incorporated our techniques and achieved a noticeable 5.8 and 14.2 improvement in

关键词： Autonomous driving 3d semantic segmentation lidar point clouds sensor fusion spherical projection noise analysis convolutional neural networks

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：