检索结果-内蒙古大学图书馆

unsupervised semantic segmentation of Urban Scenes via Cross-Modal Distillation

INTERNATIONAL JOURNAL OF COMPUTER VISION 2025年第6期133卷 3519-3541页

作者： Vobecky, Antonin Hurych, David Simeoni, Oriane Gidaris, Spyros Bursuc, Andrei Perez, Patrick Sivic, Josef Czech Tech Univ Czech Inst Informat Robot & Cybernet Prague Czech Republic Valeo Ai Paris France Czech Tech Univ Fac Elect Engn Prague Czech Republic Kyutai Paris France

semantic image segmentation models typically require extensive pixel-wise annotations, which are costly to obtain and prone to biases. Our work investigates learning semantic segmentation in urban scenes without any manual annotation. We propose a novel method for learning pixel-wise semantic segmentation using raw, uncurated data from vehicle-mounted cameras and LiDAR sensors, thus eliminating the need for manual labeling. Our contributions are as follows. First, we develop a novel approach for cross-modal unsupervised learning of semantic segmentation by leveraging synchronized LiDAR and image data. A crucial element of our method is the integration of an object proposal module that examines the LiDAR point cloud to generate proposals for spatially consistent objects. Second, we demonstrate that these 3D object proposals can be aligned with corresponding images and effectively grouped into semantically meaningful pseudo-classes. Third, we introduce a cross-modal distillation technique that utilizes image data partially annotated with the learnt pseudo-classes to train a transformer-based model for semantic image segmentation. Fourth, we demonstrate further significant improvements of our approach by extending the proposed model using a teacher-student distillation with an exponential moving average and incorporating soft targets from the teacher. We show the generalization capabilities of our method by testing on four different testing datasets (Cityscapes, Dark Zurich, Nighttime Driving, and ACDC) without any fine-tuning. We present an in-depth experimental analysis of the proposed model including results when using another pre-training dataset, per-class and pixel accuracy results, confusion matrices, PCA visualization, k-NN evaluation, ablations of the number of clusters and LiDAR's density, supervised finetuning as well as additional qualitative results and their analysis.

关键词： Autonomous driving unsupervised semantic segmentation Multimodal learning

来源：评论

学校读者我要写书评

暂无评论

unsupervised semantic segmentation with Feature Enhancement for Few-shot Image Classification 10

Unsupervised Semantic Segmentation with Feature Enhancement ...

引用

10th International Conference on Advanced Cloud and Big Data (CBD)

作者： Li, Xiang Xu, Zhuoming Xu, Qi Tang, Yan Hohai Univ Coll Comp & Informat Nanjing Peoples R China

ISBN: (纸本)9798350309713

Image classification is a typical task in big data applications. As a few-shot learning (FSL) task, the few-shot image classification attempts to learn a new visual concept from limited labelled images. The existing few-shot image classification methods usually fail to effectively eliminate the interference of image background information, thus affecting the accuracy of image classification. To overcome the shortcomings of existing methods and eliminate the interference of background information, this paper proposes a novel method named unsupervised semantic segmentation with Feature Enhancement (USSFE) for few-shot image classification. USSFE uses the dense contrastive learning to train a robust unsupervised semantic segmentation model for foreground segmentation, and enhances feature representations of the foreground feature maps by extracting the shape and texture features of the foreground images. Our experimental results on two benchmark datasets, miniImageNet and tieredImageNet, show that the proposed USSFE method achieves better performance than the state-of-the-art methods.

关键词： few-shot image classification few-shot learning N-way K-shot metric-based learning unsupervised semantic segmentation feature enhancement

来源：评论

学校读者我要写书评

暂无评论

unsupervised semantic segmentation of radar sounder data using contrastive learning 28

Unsupervised semantic segmentation of radar sounder data usi...

引用

Conference on Image and Signal Processing for Remote Sensing XXVIII

作者： Donini, Elena Amico, Mattia Bruzzone, Lorenzo Bovolo, Francesca Fdn Bruno Kessler Ctr Digital Soc Via Sommarive 18 I-38123 Trento Italy Univ Trento Dept Informat Engn & Comp Sci Sommarive 5 I-38123 Trento Italy

ISBN: (纸本)9781510655386;9781510655379

Radar Sounders (RSs) are active sensors widely used for planetary exploration and Earth observation that probe the subsurface in a non-intrusive way by acquiring vertical profiles, called radargrams. Radargrams contain information on subsurface geology and are analyzed with neural networks for segmentation and target detection. However, most of these methods rely on supervised training, which requires a large amount of labeled data that is hard to retrieve. Hence, a need emerges for a novel method for unsupervised radargram segmentation. This paper proposes a novel method for unsupervised radargram segmentation by analyzing semantically meaningful features extracted from a deep network trained with a contrastive logic. First, the network (encoder) is trained using a pretext task to extract meaningful features (query). Considering a dictionary of possible features (keys), the encoder training loss can be defined as a dictionary look-up problem. Each query is matched to a key in a large and consistent dictionary. Although such a dictionary is not available for RS data, it is dynamically computed by extracting meaningful features with another deep network called the momentum encoder. Secondly, deep feature vectors are extracted from the encoder for all radargram pixels. After the feature selection, the feature vectors are binarized. Since pixels of the same class are expected to have similar feature vectors, we compute the similarity between the feature vectors to generate a cluster of pixels for each class. We applied the proposed method to segment radargrams acquired in Greenland by the MCoRDS-3 sensor, achieving good overall accuracy.

关键词： unsupervised semantic segmentation Contrastive learning Image processing Radar sounder Subsurface sensing

来源：评论

学校读者我要写书评

暂无评论

unsupervised semantic segmentation OF KIDNEYS USING RADIAL TRANSFORM SAMPLING ON LIMITED IMAGES

UNSUPERVISED SEMANTIC SEGMENTATION OF KIDNEYS USING RADIAL T...

引用

IEEE Global Conference on Signal and Information Processing (GlobalSIP)

作者： Salehinejad, Hojjat Naqvi, Sumeya Colak, Errol Barfett, Joseph Valaee, Shahrokh Univ Toronto Dept Elect & Comp Engn Toronto ON Canada Univ Toronto St Michaels Hosp Dept Med Imaging Toronto ON Canada

ISBN: (纸本)9781728112954

Efficient training of supervised deep learning models for semantic segmentation requires a massive volume of annotated data. In this paper, we propose an unsupervised semantic segmentation method through the application of a radial transform method in the polar coordinate system to unannotated images. This method generates radial transformed images up to the number of pixels in the input image. Each generated image corresponds to a pixel in the original image, which is a spatial representation of the selected pixel with respect to other pixels, in the polar coordinate system. A dimension reduction method, such as a convolutional autoencoder (CAE), can extract features of these representations for clustering and later, labeling the original image by mapping the labels back to the original image. The advantage of the proposed radial transform technique is that it generates a massive number of training images by sampling pixels in the polar coordinate system from a very limited number of original images in the Cartesian coordinate system. The proposed approach achieved 88 : 20% accuracy in pixel-level segmentation of left kidney, right kidney, and non-kidney pixels in contrast-enhanced computed tomography (CT) images.

关键词： Computed tomography kidney radial transform sampling unsupervised semantic segmentation

来源：评论

学校读者我要写书评

暂无评论

unsupervised semantic segmentation Through Depth-Guided Feature Correlation and Sampling

Unsupervised Semantic Segmentation Through Depth-Guided Feat...

引用

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Sick, Leon Engel, Dominik Hermosilla, Pedro Ropinski, Timo Ulm Univ Ulm Germany TU Vienna Vienna Austria

ISBN: (纸本)9798350353013;9798350353006

Traditionally, training neural networks to perform semantic segmentation requires expensive human-made annotations. But more recently, advances in the field of unsupervised learning have made significant progress on this issue and towards closing the gap to supervised algorithms. To achieve this, semantic knowledge is distilled by learning to correlate randomly sampled features from images across an entire dataset. In this work, we build upon these advances by incorporating information about the structure of the scene into the training process through the use of depth information. We achieve this by (1) learning depth-feature correlation by spatially correlating the feature maps with the depth maps to induce knowledge about the structure of the scene and (2) exploiting farthest-point sampling to more effectively select relevant features by utilizing 3D sampling techniques on depth information of the scene. Finally, we demonstrate the effectiveness of our technical contributions through extensive experimentation and present significant improvements in performance across multiple benchmark datasets.

关键词： semantic segmentation unsupervised Learning unsupervised semantic segmentation

来源：评论

学校读者我要写书评

暂无评论

Imbalance-Aware Discriminative Clustering for unsupervised semantic segmentation

引用

INTERNATIONAL JOURNAL OF COMPUTER VISION 2024年第10期132卷 4362-4378页

作者： Liu, Mingyuan Zhang, Jicong Tang, Wei Beihang Univ Dept Biol Sci & Med Engn Beijing 100191 Peoples R China Univ Illinois Dept Comp Sci Chicago IL 60607 USA

unsupervised semantic segmentation (USS) aims at partitioning an image into semantically meaningful segments by learning from a collection of unlabeled images. The effectiveness of current approaches is plagued by difficulties in coordinating representation learning and pixel clustering, modeling the varying feature distributions of different classes, handling outliers and noise, and addressing the pixel class imbalance problem. This paper introduces a novel approach, termed Imbalance-Aware Dense Discriminative Clustering (IDDC), for USS, which addresses all these difficulties in a unified framework. Different from existing approaches, which learn USS in two stages (i.e., generating and updating pseudo masks, or refining and clustering embeddings), IDDC learns pixel-wise feature representation and dense discriminative clustering in an end-to-end and self-supervised manner, through a novel objective function that transfers the manifold structure of pixels in the embedding space of a vision Transformer (ViT) to the label space while tolerating the noise in pixel affinities. During inference, the trained model directly outputs the classification probability of each pixel conditioned on the image. In addition, this paper proposes a new regularizer, based on the Weibull function, to handle pixel class imbalance and cluster degeneration in a single shot. Experimental results demonstrate that IDDC significantly outperforms all previous USS methods on three real-world datasets, COCO-Stuff-27, COCO-Stuff-171, and Cityscapes. Extensive ablation studies validate the effectiveness of each design. Our code is available at https://***/MY-LIU100101/IDDC.

关键词： unsupervised semantic segmentation Imbalance-Aware Dense Discriminative Clustering End-to-end training Deep clustering

来源：评论

学校读者我要写书评

暂无评论

Enhancing unsupervised semantic segmentation Through Context-Aware Clustering

引用

IEEE TRANSACTIONS ON MULTIMEDIA 2024年 26卷 10081-10093页

作者： Zhuo, Wei Wang, Yuan Chen, Junliang Deng, Songhe Wang, Zhi Shen, Linlin Zhu, Wenwu Shenzhen Univ Natl Engn Lab Big Data Syst Comp Technol Shenzhen 518060 Peoples R China Tsinghua Univ Tsinghua Berkeley Shenzhen Inst Beijing 100190 Peoples R China Shenzhen Univ Coll Comp Sci & Software Engn Shenzhen 518060 Peoples R China Tsinghua Univ Shenzhen Int Grad Sch Beijing 100190 Peoples R China Shenzhen Univ Coll Comp Sci & Software Engineer ing Natl Engn Lab Big Data Syst Comp Technol Shenzhen 518060 Peoples R China Univ Nottingham Ningbo China Dept Comp Sci Ningbo 315100 Peoples R China Tsinghua Univ Dept Comp Sci & Technol Beijing 100190 Peoples R China

Despite the great progress of semantic segmentation with supervised learning, annotating large amounts of pixel-wise labels is, however, very expensive and time-consuming. To this end, unsupervised semantic segmentation(USS) has been proposed to learn semantic segmentation, without any form of annotations. This approach involves dense prediction of semantics which is however challenging due to the unreliable nature of local representations. To solve this problem, we propose a newly context-aware unsupervised semantic segmentation framework, which aims to enhance the unsupervised semantic segmentation by leveraging contextual knowledge within and across images. In particular, we introduce a training strategy based on our Pyramid semantic Guidance (PSG), which utilizes holistic semantics on pyramid views to guide pixel clustering with a siamese network-based framework. Additionally, we introduce a Context-Aware Embedding (CAE) module to fuse global features with low-level geometrical and appearance representations. We evaluate our method on the COCO-Stuff dataset and achieved competitive results compared to both the convolutional and ViT-based USS methods. Specifically, we attain significant improvements of +4.5% and +5% mIoU for Stuff and all class segmentation respectively, compared to previous approaches that employ unsupervised convolutional backbones.

关键词： semantic segmentation semantics Training Annotations unsupervised learning Convolutional neural networks Computer science unsupervised semantic segmentation self-supervised learning semantic clustering pseudo labeling context-aware embedding

来源：评论

学校读者我要写书评

暂无评论

Learn to Rectify the Bias of CLIP for unsupervised semantic segmentation

Learn to Rectify the Bias of CLIP for Unsupervised Semantic ...

引用

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Wang, Jingyun Kang, Guoliang Beihang Univ Beijing Peoples R China Beihang Univ Zhongguancun Lab Beijing Peoples R China

ISBN: (纸本)9798350353013;9798350353006

Recent works utilize CLIP to perform the challenging unsupervised semantic segmentation task where only images without annotations are available. However, we observe that when adopting CLIP to such a pixel-level understanding task, unexpected bias occurs. Previous works don't explicitly model such bias, which largely constrains the segmentation performance. In this paper, we propose to explicitly model and rectify the bias existing in CLIP to facilitate the unsupervised semantic segmentation. Specifically, we design a learnable "Reference" prompt to encode class-preference bias and project the positional embedding of vision transformer to represent space-preference bias. Via a simple element-wise subtraction, we rectify the logits of CLIP classifier. Based on the rectified logits, we generate a segmentation mask via a Gumbel-Softmax operation. Then a contrastive loss between masked visual feature and the text features of different classes is imposed to facilitate the effective bias modeling. To further improve the segmentation, we distill the knowledge from the rectified CLIP to the advanced segmentation architecture via minimizing our designed mask-guided, feature-guided and text-guided loss terms. Extensive experiments on standard benchmarks demonstrate that our method performs favorably against previous state-of-the-arts. The implementation is available at https://***/dogehhh/ReCLIP.

关键词： unsupervised semantic segmentation

来源：评论

学校读者我要写书评

暂无评论

Drive&Segment: unsupervised semantic segmentation of Urban Scenes via Cross-Modal Distillation 17th

Drive&Segment: Unsupervised Semantic Segmentation of Urban S...

引用

17th European Conference on Computer Vision (ECCV)

ISBN: (纸本)9783031198380;9783031198397

This work investigates learning pixel-wise semantic image segmentation in urban scenes without any manual annotation, just from the raw non-curated data collected by cars which, equipped with cameras and LiDAR sensors, drive around a city. Our contributions are threefold. First, we propose a novel method for cross-modal unsupervised learning of semantic image segmentation by leveraging synchronized LiDAR and image data. The key ingredient of our method is the use of an object proposal module that analyzes the LiDAR point cloud to obtain proposals for spatially consistent objects. Second, we show that these 3D object proposals can be aligned with the input images and reliably clustered into semantically meaningful pseudo-classes. Finally, we develop a cross-modal distillation approach that leverages image data partially annotated with the resulting pseudo-classes to train a transformer-based model for image semantic segmentation. We show the generalization capabilities of our method by testing on four different testing datasets (Cityscapes, Dark Zurich, Nighttime Driving and ACDC) without any finetuning, and demonstrate significant improvements compared to the current state of the art on this problem.

关键词： Autonomous driving unsupervised semantic segmentation

来源：评论

学校读者我要写书评

暂无评论

EAGLE: Eigen Aggregation Learning for Object-Centric unsupervised semantic segmentation

EAGLE: Eigen Aggregation Learning for Object-Centric Unsuper...

引用

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Kim, Chanyoung Han, Woojung Ju, Dayun Hwang, Seong Jae Yonsei Univ Seoul South Korea

ISBN: (纸本)9798350353013;9798350353006

semantic segmentation has innately relied on extensive pixel-level annotated data, leading to the emergence of unsupervised methodologies. Among them, leveraging self-supervised Vision Transformers for unsupervised semantic segmentation (USS) has been making steady progress with expressive deep features. Yet, for semantically segmenting images with complex objects, a predominant challenge remains: the lack of explicit object-level semantic encoding in patch-level features. This technical limitation often leads to inadequate segmentation of complex objects with diverse structures. To address this gap, we present a novel approach, EAGLE, which emphasizes object-centric representation learning for unsupervised semantic segmentation. Specifically, we introduce EiCue, a spectral technique providing semantic and structural cues through an eigenbasis derived from the semantic similarity matrix of deep image features and color affinity from an image. Further, by incorporating our object-centric contrastive loss with EiCue, we guide our model to learn object-level representations with intra- and inter-image object-feature consistency, thereby enhancing semantic accuracy. Extensive experiments on COCO-Stuff, Cityscapes, and Potsdam-3 datasets demonstrate the state-of-the-art USS results of EAGLE with accurate and consistent semantic segmentation across complex scenes.

关键词： Contrastive Learning Graph Theory unsupervised semantic segmentation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：