检索结果-内蒙古大学图书馆

MOODv2: masked image modeling for Out-of-Distribution Detection

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2024年第12期46卷 8994-9003页

作者： Li, Jingyao Chen, Pengguang Yu, Shaozuo Liu, Shu Jia, Jiaya Chinese Univ Hong Kong Dept Comp Sci & Engn Cent Ave Hong Kong Peoples R China SmartMore Shatin Hong Kong Peoples R China

The crux of effective out-of-distribution (OOD) detection lies in acquiring a robust in-distribution (ID) representation, distinct from OOD samples. While previous methods predominantly leaned on recognition-based techniques for this purpose, they often resulted in shortcut learning, lacking comprehensive representations. In our study, we conducted a comprehensive analysis, exploring distinct pretraining tasks and employing various OOD score functions. The results highlight that the feature representations pre-trained through reconstruction yield a notable enhancement and narrow the performance gap among various score functions. This suggests that even simple score functions can rival complex ones when leveraging reconstruction-based pretext tasks. Reconstruction-based pretext tasks adapt well to various score functions. As such, it holds promising potential for further expansion. Our OOD detection framework, MOODv2, employs the masked image modeling pretext task. Without bells and whistles, MOODv2 impressively enhances 14.30% AUROC to 95.68% on imageNet and achieves 99.98% on CIFAR-10.

关键词： Computer vision masked image modeling out-of-distribution detection outlier detection

来源：评论

学校读者我要写书评

暂无评论

masked image modeling Auxiliary Pseudo-Label Propagation with a Clustering Central Rectification Strategy for Cross-Scene Classification

引用

REMOTE SENSING 2024年第11期16卷 1983页

作者： Zhang, Xinyi Zhuang, Yin Zhang, Tong Li, Can Chen, He Beijing Inst Technol Natl Key Lab Sci & Technol Space Born Intelligent Beijing 100081 Peoples R China

Cross-scene classification focuses on setting up an effective domain adaptation (DA) way to transfer the learnable knowledge from source to target domain, which can be reasonably achieved through the pseudo-label propagation procedure. However, it is hard to bridge the objective existing severe domain discrepancy between source and target domains, and thus, there are several unreliable pseudo-labels generated in target domain and involved into pseudo-label propagation procedure, which would lead to unreliable error accumulation to deteriorate the performance of cross-scene classification. Therefore, in this paper, a novel masked image modeling Auxiliary Pseudo-Label Propagation called MIM-AP2 with clustering central rectification strategy is proposed to improve the quality of pseudo-label propagation for cross-scene classification. First, in order to gracefully bridge the domain discrepancy and improve DA representation ability in-domain, a supervised class-token contrastive learning is designed to find the more consistent contextual clues to achieve knowledge transfer learning from source to target domain. At the same time, it is also incorporated with a self-supervised MIM mechanism according to a low random masking ratio to capture domain-specific information for improving the discriminability in-domain, which can lay a solid foundation for high-quality pseudo-label generation. Second, aiming to alleviate the impact of unreliable error accumulation, a clustering central rectification strategy is designed to adaptively update robustness clustering central representations to assist in rectifying unreliable pseudo-labels and learning a superior target domain specific classifier for cross-scene classification. Finally, extensive experiments are conducted on six cross-scene classification benchmarks, and the results are superior to other DA methods. The average accuracy reached 95.79%, which represents a 21.87% improvement over the baseline. This demonstrates that the

关键词： cross-scene classification domain adaptation masked image modeling pseudo-label

来源：评论

学校读者我要写书评

暂无评论

masked image modeling-based boundary reconstruction for 3D medical image segmentation

引用

COMPUTERS IN BIOLOGY AND MEDICINE 2023年 166卷 107526-107526页

作者： Liu, Chang Cheng, Yuanzhi Tamura, Shinichi Harbin Inst Technol Sch Comp Sci & Technol Harbin 150001 Peoples R China Qingdao Univ Sci & Technol Sch Informat Sci & Technol Qingdao 266061 Shandong Peoples R China Osaka Univ Grad Sch Med 2-2 Yamadaoka Suita Osaka 5650871 Japan

Accurate segmentation of 3D medical images is vital for computer-aided diagnosis. However, the complexity of target morphological variations and a scarcity of labeled data make segmentation more challenging. Furthermore, existing models make it difficult to fully and efficiently integrate global and local information, which hinders structured knowledge acquisition. To overcome these challenges, we introduce the TNT Masking Network (TNT-MNet), a groundbreaking transformer-based 3D model that utilizes a transformer-in-transformer (TNT) encoder. For the first time, we present masked image modeling (MIM) in supervised learning, utilizing target boundary regions as masked prediction targets to enhance structured knowledge acquisition. We execute multiscale random masking on inner and outer tokens in online branch to tackle the challenge of segmenting organs and lesion regions with varying structures at multiple scales and to enhance modeling capabilities. In contrast, the target branch utilizes all tokens to guide the online branch to reconstruct the masked tokens. Our experiments suggest that TNT-MNet's performance is comparable, or even better, than state-of-the-art models in three medical image datasets (BTCV, LiTS2017, and BraTS2020) and effectively reduces the dependence on labeled data. The code and models are publicly available at https://***/changliu-work/TNT_MNet.

关键词： Medical image segmentation masked image modeling Online-target two-branch network Boundary prediction Supervised training

来源：评论

学校读者我要写书评

暂无评论

MIMIC: masked image modeling with image Correspondences

MIMIC: Masked Image Modeling with Image Correspondences

引用

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Marathe, Kalyani Bigverdi, Mahtab Khan, Nishat Kundu, Tuhin Howe, Patrick Ranjit, Sharan S. Bhattad, Anand Kembhavi, Aniruddha Shapiro, Linda G. Krishna, Ranjay Univ Washington Seattle WA 98195 USA Allen Inst Artificial Intelligence Seattle WA USA Toyota Technol Inst Chicago IL USA

ISBN: (纸本)9798350365474

Dense pixel-specific representation learning at scale has been bottlenecked due to the unavailability of large-scale multi-view datasets. Current methods for building effective pretraining datasets heavily rely on annotated 3D meshes, point clouds, and camera parameters from simulated environments, preventing them from building datasets from real-world data sources where such metadata is lacking. We introduce a pretraining dataset-curation approach that does not require any additional annotations. Our method allows us to generate multi-view datasets from both realworld videos and simulated environments at scale. Specifically, we experiment with two scales: MIMIC-1M with 1.3M and MIMIC- 3M with 3.1M multi-view image pairs and train models with different masked image modeling objectives. Through our comprehensive experimental analysis we show that: Representations trained on our automatically generated MIMIC- 3M outperform those learned from expensive crowdsourced datasets (imageNet-1K) and those learned from synthetic environments (MULTIVIEW-HABITAT) on three dense geometric tasks: depth estimation on NYUv2 (up arrow 1.7%), and surface normal estimation on Taskonomy (down arrow 2.05%), and depth estimation on Taskonomy (down arrow 7.5%) and performs on-par with MULTIVIEW-HABITAT on Taskonomy edges and curvature tasks. Larger dataset (MIMIC-3M) improves performance, which is promising since our curation method can arbitrarily scale to produce even larger datasets. The code and instructions to download, access, and use MIMIC- 3M can be found here.

关键词： Dense representation learning masked image modeling

来源：评论

学校读者我要写书评

暂无评论

PYRAMID masked image modeling FOR TRANSFORMER-BASED AERIAL OBJECT DETECTION 30

PYRAMID MASKED IMAGE MODELING FOR TRANSFORMER-BASED AERIAL O...

引用

30th IEEE International Conference on image Processing (ICIP)

作者： Zhang, Cong Liu, Tianshan Ju, Yakun Lam, Kin-Man Hong Kong Polytech Univ Dept Elect & Informat Engn Kowloon Hong Kong Peoples R China

ISBN: (纸本)9781728198354

Two obstacles, the scarcity of annotated samples and the difficulty in preserving multi-scale hierarchical representations, hinder the advancement of vision Transformer-based aerial object detection. The emergence of self-supervised learning has inspired some solutions to the first issue. However, most solutions focus on single-scale features, conflicting with solving the second issue. To bridge this gap, this paper proposes a novel pyramid masked image modeling (MIM) framework, termed PyraMIM, for self-supervised pretraining in aerial scenarios. Without manual annotation, PyraMIM enables establishing pyramid representations during pretraining, which can be seamlessly adapted to downstream aerial object detection for performance improvement. Experimental results demonstrate the effectiveness and superiority of our method.

关键词： Vision Transformer masked image modeling Self-Supervised Learning Pyramid Architecture Aerial Object Detection

来源：评论

学校读者我要写书评

暂无评论

ENHANCED masked image modeling FOR ANALYSIS OF DENTAL PANORAMIC RADIOGRAPHS 20

ENHANCED MASKED IMAGE MODELING FOR ANALYSIS OF DENTAL PANORA...

引用

20th IEEE International Symposium on Biomedical Imaging (ISBI)

作者： Almalki, Amani Latecki, Longin Jan Temple Univ Dept Comp & Informat Sci Philadelphia PA 19122 USA

ISBN: (纸本)9781665473583

The computer-assisted radiologic informative report has received increasing research attention to facilitate diagnosis and treatment planning for dental care providers. However, manual interpretation of dental images is limited, expensive, and time-consuming. Another barrier in dental imaging is the limited number of available images for training, which is a challenge in the era of deep learning. This study proposes a novel self-distillation (SD) enhanced self-supervised learning on top of the masked image modeling (SimMIM) Transformer, called SD-SimMIM, to improve the outcome with a limited number of dental radiographs. In addition to the prediction loss on masked patches, SD-SimMIM computes the self-distillation loss on the visible patches. We apply SDSimMIM on dental panoramic X-rays for teeth numbering, detection of dental restorations and orthodontic appliances, and instance segmentation tasks. Our results show that SDSimMIM outperforms other self-supervised learning methods. Furthermore, we augment and improve the annotation of an existing dataset of panoramic X-rays.

关键词： Self-distillation Self-supervised learning masked image modeling Object detection Instance segmentation

来源：评论

学校读者我要写书评

暂无评论

Information-density Masking Strategy for masked image modeling

Information-density Masking Strategy for Masked Image Modeli...

引用

IEEE International Conference on Multimedia and Expo (ICME)

作者： Zhu, He Chen, Yang Hu, Guyue Yu, Shan Chinese Acad Sci CASIA Inst Automat Brainnetome Ctr Natl Lab Pattern Recognit NLPR Beijing Peoples R China Univ Chinese Acad Sci UCAS Sch Future Technol Beijing Peoples R China Nanyang Technol Univ Sch Comp Sci & Engn Singapore Singapore Univ Chinese Acad Sci UCAS Sch Artificial Intelligence Beijing Peoples R China

ISBN: (纸本)9781665468916

Recent representation learning approaches mainly fall into two paradigms: contrastive learning (CL) and masked image modeling (MIM). Combining these two methods may boost the performance, but its learning process still heavily depends on the random masking strategy. We conjecture that the random masking may hinder learning the comprehensive relationship between concept and visual patches. To overcome these limitations, we propose an information-density masking (IDM) strategy for general visual transformers. Specifically, the IDM mask out the visual patches according to their activation values of attention maps. To obtain the attention maps before the reconstruction, a self-supervised training framework CAMAE is further proposed. In addition, in order to reduce the redundancy among different attention maps, we introduce a pattern-learning balance (PLB) sampling to adaptively adjust the learning progress in different attention spaces. Extensive experiments indicate that our method efficiently retains more comprehensive visual characteristics and achieves state-of-the-art performance.

关键词： masked image modeling contrastive learning unsupervised learning

来源：评论

学校读者我要写书评

暂无评论

When masked image modeling Meets Source-free Unsupervised Domain Adaptation: Dual-Level masked Network for Semantic Segmentation 23

When Masked Image Modeling Meets Source-free Unsupervised Do...

引用

31st ACM International Conference on Multimedia (MM)

作者： Li, Gang Ma, Xianzheng Wang, Zhao Li, Hao Zhang, Qifei Wu, Chao Zhejiang Univ Ningbo Peoples R China Shanghai AI Lab Shanghai Peoples R China

ISBN: (纸本)9798400701085

Source-Free domain adaptive Semantic Segmentation (SFSS) aims to transfer knowledge from source domain to the target domain with only pre-trained source segmentation model and the unlabeled target dataset. Only a few works have been researched for SFSS, relying on entropy minimization, pseudo-labeling. Nevertheless, due to the domain bias, these methods tend to suffering from the confusion of classes with a similar visual appearance in different domains. To address the above issue, we propose to enhance discriminability towards target samples with masked image modeling to model spatial context relations as additional recognition clues. Specifically, we design a novel Dual-Level masked Consistency method, which explicitly encourages the model to learn comprehensive context relations, i.e. patch-wise context and channel-wise context, on the target domain. By randomly masking target images and forcing the model to reconstruct predictions of the entire image with left unmasked part, the model has to make full use of spatially contextual information. To take a step further, we propose a novel masking strategy considering both local context and global context information by applying patch-wise masking on image patches and channel-wise masking on latent features. Notably, patch-wise context learning and channel-wise context learning can complement each other. Extensive experiments demonstrate the effectiveness of our proposed method and our method achieves state-of-the-art performance on two synthetic-to-real benchmarks: GTA5 -> Cityscapes and SYNTHIA -> Cityscapes.

关键词： source-free domain adaptation masked image modeling semantic segmentation

来源：评论

学校读者我要写书评

暂无评论

ATTENTION-GUIDED CONTRASTIVE masked image modeling FOR TRANSFORMER-BASED SELF-SUPERVISED LEARNING 30

ATTENTION-GUIDED CONTRASTIVE MASKED IMAGE MODELING FOR TRANS...

引用

30th IEEE International Conference on image Processing (ICIP)

作者： Zhan, Yucheng Zhao, Yucheng Luo, Chong Zhang, Yueyi Sun, Xiaoyan Univ Sci & Technol China Hefei Peoples R China Microsoft Res Asia Beijing Peoples R China

ISBN: (纸本)9781728198354

Self-supervised learning with vision transformer (ViT) has gained much attention recently. Most existing methods rely on either contrastive learning or masked image modeling. The former is suitable for global feature extraction but underperforms in fine-grained tasks. The later explores the internal structure of images but ignores the high information sparsity and unbalanced information distribution. In this paper, we propose a new approach called Attention-guided Contrastive masked image modeling (ACoMIM), which integrates the merits of both paradigms and leverages the attention mechanism of ViT for effective representation. Specifically, it has two pretext tasks, predicting the features of masked regions guided by attention and comparing the global features of masked and unmasked images. We show that these two pretext tasks complement each other and improve our method's performance. The experiments demonstrate that our model transfers well to various downstream tasks such as classification and object detection. Code is available at https://***/yczhan/ACoMIM.

关键词： Self-Supervised Learning Vision transformer masked image modeling

来源：评论

学校读者我要写书评

暂无评论

SPARSE ANATOMICAL PROMPT SEMI-SUPERVISED LEARNING WITH masked image modeling FOR CBCT TOOTH SEGMENTATION 21

SPARSE ANATOMICAL PROMPT SEMI-SUPERVISED LEARNING WITH MASKE...

引用

21st IEEE International Symposium on Biomedical Imaging (ISBI)

作者： Dai, Pengyu Ou, Yafei Yang, Yuqiao Liu, Yang Zhao, Yue Chongqing Univ Posts & Telecommun Sch Commun & Informat Engn Chongqing 400065 Peoples R China Tokyo Inst Technol Inst Innovat Res Yokohama Kanagawa 2268503 Japan Chongqing Med Univ Stomatol Hosp Chongqing 401147 Peoples R China Zhejiang Univ Sch Mech Engn Hangzhou 310058 Zhejiang Peoples R China

ISBN: (纸本)9798350313345;9798350313338

Accurate tooth identification and segmentation in Cone Beam Computed Tomography (CBCT) dental images can significantly enhance the efficiency and precesion of manual diagnoses performed by dentists. However, existing segmentation methods are mainly developed based on large data volumes training, on which their annotations are extremely time-consuming. Meanwhile, the teeth of each class in CBCT dental images being closely positioned, coupled with subtle inter-class differences, gives rise to the challenge of indistinct boundaries when training model with limited data. To address these challenges, this study aims to propose a task-oriented masked Auto-Encoder paradigm to effectively utilize large amounts of unlabeled data to achieve accurate tooth segmentation with limited labeled data. Specifically, we first construct a self-supervised pre-training framework of masked auto encoder to efficiently utilize unlabeled data to enhance the network performance. Subsequently, we introduce a sparse masked prompt mechanism based on graph attention to incorporate boundary information of the teeth, aiding the network in learning the anatomical structural features of teeth. To the best of our knowledge, we are pioneering the integration of the mask pre-training paradigm into the CBCT tooth segmentation task. Extensive experiments demonstrate both the feasibility of our proposed method and the potential of the boundary prompt mechanism.

关键词： CBCT tooth segmentation Self-supervised learning masked image modeling Semantic segmentation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：