检索结果-内蒙古大学图书馆

Label-efficient transformer-based framework with self-supervised strategies for heterogeneous lung tumor segmentation

引用

EXPERT SYSTEMS WITH APPLICATIONS 2025年 269卷

作者： Liu, Zhenbing Li, Weixing Cui, Yanfen Chen, Xin Pan, Xipeng Ye, Guanchao Wu, Guangyao Liao, Yongde Volmer, Leroy Wee, Leonard Dekker, Andre Han, Chu Liu, Zaiyi Shi, Zhenwei Guilin Univ Elect Technol Sch Comp & Informat Secur Guilin 541004 Peoples R China Southern Med Univ Guangdong Prov Peoples Hosp Guangdong Acad Med Sci Dept Radiol Guangzhou 510080 Peoples R China Guangdong Prov Key Lab Artificial Intelligence Med Guangzhou 510080 Peoples R China Southern Med Univ Guangdong Prov Peoples Hosp Med Res Inst Guangdong Acad Med Sci Guangzhou 510080 Peoples R China South China Univ Technol Guangzhou Peoples Hosp 1 Sch Med Dept Radiol Guangzhou 510080 Peoples R China Shanxi Med Univ Shanxi Prov Canc Hosp Shanxi HospCanc Hosp Chinese Acad Med SciDept Radiol Taiyuan 030013 Peoples R China Huazhong Univ Sci & Technol Union Hosp Tongji Med Coll Dept Thorac Surg Wuhan 430021 Peoples R China Huazhong Univ Sci & Technol Union Hosp Tongji Med Coll Dept Radiol Wuhan 430021 Peoples R China Maastricht Univ Fac Hlth Med Life Sci Clin Data Sci NL-6229 ET Maastricht Netherlands Maastricht Univ GROW Sch Oncol & Reprod Dept Radiat Oncol Maastro Med Ctr NL-6229 ET Maastricht Netherlands

Precise and automatic segmentation of lung tumors is crucial for computer-aided diagnosis and subsequent treatment planning. However, the heterogeneity of lung tumors, varying in size, shape, and location, combined with the low contrast between tumors and adjacent tissues, significantly complicates accurate segmentation. Furthermore, most supervised segmentation models face limitations due to the scarcity and lack of diversity in labeled training data. Although various self-supervised learning strategies have been developed for model pre-training with unlabeled data, their relative benefits for the downstream task of lung tumor segmentation on CT scans remain uncertain. To address these challenges, we introduce a robust and label-efficient Transformer- based framework with different self-supervised strategies for lung tumor segmentation. Our model training is conducted in two phase, during the pre-training phase, we pre-train the model on a large amount of unlabeled CT scans, employing three different pre-training strategies and comparing their impacts on downstream lung tumor segmentation task. In the fine-tuning phase, we utilize the encoders of the pre-trained models for label-efficient supervised fine-tuning. In addition, we design a surrounding samples-based contrastive learning (SSCL) module at the end of the encoder to enhance feature extraction, especially for tumors with indistinct boundaries. Our proposed methods are evaluated on test sets from seven different center. When only a small amount of labeled data is available, compared to supervised models, Ours (SimMIM3D) demonstrates superior segmentation performance on three internal test sets, achieving Dice coefficients of 0.8419, 0.8346, and 0.8282, respectively. Additionally, it also shows strong generalization on external test sets, with Dice coefficients of 0.7594, 0.7684, 0.6578, and 0.6621, respectively. Extensive experiments confirm the efficacy of our methodology, demonstrating significant improvem

关键词： Lung tumor segmentation Vision transformer Contrastive learning masked image modeling Self-supervised learning

来源：评论

学校读者我要写书评

暂无评论

Real face foundation representation learning for generalized deepfake detection

引用

PATTERN RECOGNITION 2025年 161卷

作者： Shi, Liang Zhang, Jie Ji, Zhilong Bai, Jinfeng Shan, Shiguang Chinese Acad Sci Inst Comp Technol Key Lab AI Safety CAS Beijing 100090 Peoples R China Univ Chinese Acad Sci Beijing 100049 Peoples R China Chinese Acad Sci Inst Intelligent Comp Technol Suzhou 215124 Peoples R China Tomorrow Adv Life Educ Grp Beijing 100085 Peoples R China

The emergence of deepfake technologies has become a matter of social concern as they pose threats to individual privacy and public security. It is now of great significance to develop reliable deepfake detectors. However, with numerous face manipulation algorithms present, existing detectors fail to generalize to all types of manipulated faces. To address this, we propose an alternative method that primarily models the distribution of real faces. Our approach, named Real Face Foundation Representation Learning (RFFR), aims to learn a general representation from large-scale real face datasets and identifies potential artifacts that deviate from the distribution of RFFR. Specifically, we train a model on real face datasets by masked image modeling (MIM). When applying the model on fake samples, we observe clear discrepancies between input faces and the reconstructed ones, which reveals artifacts absent in the RFFR distribution. However, these discrepancies are significantly less evident in real faces, which makes it easier to build a deepfake detector sensitive to a wide range of potential artifacts. Extensive experiments demonstrate that our method achieves better generalization performance, as it significantly outperforms the state-of-the-art methods in cross-manipulation evaluations, and has the potential to further improve by introducing extra real faces for training RFFR.

关键词： Deepfake detection masked image modeling Representation learning Vision Transformers

来源：评论

学校读者我要写书评

暂无评论

MambaMIM: Pre-training Mamba with state space token interpolation and its application to medical image segmentation

引用

Medical image analysis 2025年 103卷 103606页

作者： Fenghe Tang Bingkun Nian Yingtai Li Zihang Jiang Jie Yang Wei Liu S Kevin Zhou School of Biomedical Engineering Division of Life Sciences and Medicine University of Science and Technology of China Hefei Anhui 230026 PR China Suzhou Institute for Advanced Research University of Science and Technology of China Suzhou Jiangsu 215123 PR China Center for Medical Imaging Robotics and Analytic Computing & LEarning (MIRACLE) Suzhou Institute for Advanced Research USTC Suzhou Jiangsu 215123 PR China. Department of Automation Shanghai Jiao Tong University Shanghai PR China Institute of Image Processing and Pattern Recognition Shanghai PR China Institute of Medical Robotics Shanghai Jiao Tong University Shanghai PR China. Institute of Medical Robotics Shanghai Jiao Tong University Shanghai PR China. Electronic address: weiliucv@***. Center for Medical Imaging Robotics and Analytic Computing & LEarning (MIRACLE) Suzhou Institute for Advanced Research USTC Suzhou Jiangsu 215123 PR China Jiangsu Provincial Key Laboratory of Multimodal Digital Twin Technology Suzhou Jiangsu 215123 China State Key Laboratory of Precision and Intelligent Chemistry University of Science and Technology of China Hefei Anhui 230026 China. Electronic address: skevinzhou@***.

Recently, the state space model Mamba has demonstrated efficient long-sequence modeling capabilities, particularly for addressing long-sequence visual tasks in 3D medical imaging. However, existing generative self-supervised learning methods have not yet fully unleashed Mamba's potential for handling long-range dependencies because they overlook the inherent causal properties of state space sequences in masked modeling. To address this challenge, we propose a general-purpose pre-training framework called MambaMIM, a masked image modeling method based on a novel TOKen-Interpolation strategy (TOKI) for the selective structure state space sequence, which learns causal relationships of state space within the masked sequence. Further, MambaMIM introduces a bottom-up 3D hybrid masking strategy to maintain a masking consistency across different architectures and can be used on any single or hybrid Mamba architecture to enhance its multi-scale and long-range representation capability. We pre-train MambaMIM on a large-scale dataset of 6.8K CT scans and evaluate its performance across eight public medical segmentation benchmarks. Extensive downstream experiments reveal the feasibility and advancement of using Mamba for medical image pre-training. In particular, when we apply the MambaMIM to a customized architecture that hybridizes MedNeXt and Vision Mamba, we consistently obtain the state-of-the-art segmentation performance. The code is available at: https://***/FengheTan9/MambaMIM.

关键词： masked image modeling Medical image pre-training Medical image segmentation State space token interpolation

来源：评论

学校读者我要写书评

暂无评论

An effective masked transformer network for image denoising

引用

SIGNAL image AND VIDEO PROCESSING 2024年第6-7期18卷 4997-5010页

作者： Xu, Shaoping Xiao, Nan Tao, Wuyong Zhou, Changfei Xiong, Minghai Nanchang Univ Sch Math & Comp Sci Nanchang 330031 Peoples R China

The rising popularity of employing deep learning networks for image denoising can be observed over the past decade. Typically, their exceptional performance is rooted in their ability to learn the mapping from noisy images to clear ones through extensive training on image datasets. However, variations in noise types and intensity between test and training images significantly impact their performance. Hence, due to its weak generalization capability in practical applications, it becomes imperative to train multiple denoising models for different types of noise and varying degrees of noise interference. To address this challenge, we introduce an effective masked Transformer network by incorporating the random mask module into the network. Specifically, a random mask module is integrated into the Transformer to randomly discard certain features, thereby augmenting the network's generalization capabilities. Furthermore, the random mask module is also applied during the input processing stage to reduce reliance on the whole original image information. Additionally, the sampling operations are added to the network. Based on this, the image size is reduced to half of the original size through downsampling layer, significantly improving the execution efficiency. Experimental results manifest that the proposed masked Transformer network surpasses state-of-the-art methods like SwinIR and Restormer concerning denoising effectiveness and execution efficiency across both synthetic and real-world noisy images.

关键词： image denoising Data bias masked image modeling masked Transformer Poisson noise Speckle noise

来源：评论

学校读者我要写书评

暂无评论

Semantic Communication for Efficient image Transmission Tasks based on masked Autoencoders 98

Semantic Communication for Efficient Image Transmission Task...

引用

98th IEEE Vehicular Technology Conference (VTC-Fall)

作者： Wu, Jiale Wu, Celimuge Lin, Yangfei Bao, Jingjing Du, Zhaoyang Zhong, Lei Chen, Xianfu Ji, Yusheng Univ Electrocommun Tokyo Japan Toyota Motor Co Ltd Tokyo Japan VTT Tech Res Ctr Finland Oulu Finland Natl Inst Informat NII Tokyo Japan

ISBN: (纸本)9798350329285

Semantic communication, a promising candidate for 6G technology, has become a research hot spot. However, existing studies tend to focus more on image reconstruction rather than accurately transmitting semantic information at the pixel level. This paper introduces a novel approach using codec-based masked AutoEncoders (MAE) for efficient image transmission. The proposed system compresses local information into low-dimensional latent vectors, improving system efficiency. We also design a selective module for enhanced image reconstruction and implement Noise Adversarial Training (NAT) to increase the system's resilience to channel noise. Experimental results show that our method effectively improves downstream tasks while preserving image quality.

关键词： semantic communication masked image modeling generative models deep learning

来源：评论

学校读者我要写书评

暂无评论

Adaptive-Masking Policy with Deep Reinforcement Learning for Self-Supervised Medical image Segmentation

Adaptive-Masking Policy with Deep Reinforcement Learning for...

引用

IEEE International Conference on Multimedia and Expo (ICME)

作者： Xu, Gang Wang, Shengxin Lukasiewicz, Thomas Xu, Zhenghua Hebei Univ Technol Sch Artificial Intelligence Tianjin Peoples R China Hebei Univ Technol Sch Hlth Sci & Biomed Engn State Key Lab Reliabil & Intelligence Elect Equip Tianjin Peoples R China TU Wien Inst Log & Computat Vienna Austria Univ Oxford Dept Comp Sci Oxford England

ISBN: (纸本)9781665468916

Although self-supervised learning methods based on masked image modeling have achieved some success in improving the performance of deep learning models, these methods have difficulty in ensuring that the masked region is the most appropriate for each image, resulting in segmentation networks that do not get the best weights in pre-training. Therefore, we propose a new adaptive-masking policy self-supervised learning method. Specifically, we model the process of masking images as a reinforcement learning problem and use the results of the reconstruction model as a feedback signal to guide the agent to learn the masking policy to select a more appropriate mask position and size for each image, helping the reconstruction network to learn more fine-grained image representation information and thus improve the downstream segmentation model performance. We conduct extensive experiments on two datasets, Cardiac and TCIA, and the results show that our approach outperforms current state-of-the-art self-supervised learning methods.

关键词： masked image modeling Self-supervised learning Deep reinforcement learning Medical image segmentation

来源：评论

学校读者我要写书评

暂无评论

Self-supervised 3D Anatomy Segmentation Using Self-distilled masked image Transformer (SMIT) 25th

Self-supervised 3D Anatomy Segmentation Using Self-distilled...

引用

25th International Conference on Medical image Computing and Computer Assisted Intervention (MICCAI)

作者： Jiang, Jue Tyagi, Neelam Tringale, Kathryn Crane, Christopher Veeraraghavan, Harini Mem Sloan Kettering Canc Ctr Dept Med Phys New York NY 10021 USA Mem Sloan Kettering Canc Ctr Dept Radiat Oncol 1275 York Ave New York NY 10021 USA

ISBN: (纸本)9783031164408;9783031164392

Vision transformers efficiently model long-range context and thus have demonstrated impressive accuracy gains in several image analysis tasks including segmentation. However, such methods need large labeled datasets for training, which is hard to obtain for medical image analysis. Self-supervised learning (SSL) has demonstrated success in medical image segmentation using convolutional networks. In this work, we developed a self-distillation learning with masked image modeling method to perform SSL for vision transformers (SMIT) applied to 3D multi-organ segmentation from CT and MRI. Our contribution combines a dense pixel-wise regression pretext task performed within masked patches called masked image prediction with masked patch token distillation to pre-train vision transformers. Our approach is more accurate and requires fewer fine tuning datasets than other pretext tasks. Unlike prior methods, which typically used image sets arising from disease sites and imaging modalities corresponding to the target tasks, we used 3,643 CT scans (602,708 images) arising from head and neck, lung, and kidney cancers as well as COVID-19 for pre-training and applied it to abdominal organs segmentation from MRI pancreatic cancer patients as well as publicly available 13 different abdominal organs segmentation from CT. Our method showed clear accuracy improvement (average DSC of 0.875 from MRI and 0.878 from CT) with reduced requirement for fine-tuning datasets over commonly used pretext tasks. Extensive comparisons against multiple current SSL methods were done. Our code is available at: https://***/harveerar/***.

关键词： Self-supervised learning Segmentation Self-distillation masked image modeling masked embedding transformer

来源：评论

学校读者我要写书评

暂无评论

MAPSeg: Unified Unsupervised Domain Adaptation for Heterogeneous Medical image Segmentation Based on 3D masked Autoencoding and Pseudo-Labeling

MAPSeg: Unified Unsupervised Domain Adaptation for Heterogen...

引用

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Zhang, Xuzhe Wu, Yuhao Angelini, Elsa Li, Ang Guo, Jia Rasmussen, Jerod M. O'Connor, Thomas G. Wadhwa, Pathik D. Jackowski, Andrea Parolin Li, Hai Posner, Jonathan Laine, Andrew F. Wang, Yun Columbia Univ New York NY 10027 USA Duke Univ Durham NC 27706 USA Inst Polytech Paris Telecom Paris LTCI Paris France Univ Maryland Coll Pk College Pk MD USA Univ Calif Irvine Irvine CA 92717 USA Univ Rochester Rochester NY 14627 USA Univ Fed Sao Paulo Sao Paulo Brazil Emory Univ Atlanta GA 30322 USA

ISBN: (纸本)9798350353013;9798350353006

Robust segmentation is critical for deriving quantitative measures from large-scale, multi-center, and longitudinal medical scans. Manually annotating medical scans, however, is expensive and labor-intensive and may not always be available in every domain. Unsupervised domain adaptation (UDA) is a well-studied technique that alleviates this label-scarcity problem by leveraging available labels from another domain. In this study, we introduce masked Autoencoding and Pseudo-Labeling Segmentation (MAPSeg), a unified UDA framework with great versatility and superior performance for heterogeneous and volumetric medical image segmentation. To the best of our knowledge, this is the first study that systematically reviews and develops a framework to tackle four different domain shifts in medical image segmentation. More importantly, MAPSeg is the first framework that can be applied to centralized, federated, and test-time UDA while maintaining comparable performance. We compare MAPSeg with previous state-of-the-art methods on a private infant brain MRI dataset and a public cardiac CT-MRI dataset, and MAPSeg outperforms others by a large margin (10.5 Dice improvement on the private MRI dataset and 5.7 on the public CT-MRI dataset). MAPSeg poses great practical value and can be applied to real-world problems. GitHub: https://***/XuzheZ/MAPSeg/.

关键词： generalization image segmentation masked image modeling medical image self-supervised learning unsupervised domain adaptation

来源：评论

学校读者我要写书评

暂无评论

PATCH INHERENT FEATURE GUIDED MASK SELECTION FOR image COMPRESSION 49

PATCH INHERENT FEATURE GUIDED MASK SELECTION FOR IMAGE COMPR...

引用

49th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Huang, Xianhan Zhu, Qingmeng Yu, Zhipeng Liu, Kun He, Hao Chinese Acad Sci Inst Software Beijing Peoples R China Beijing Univ Technol Beijing Peoples R China

ISBN: (纸本)9798350344868;9798350344851

Recent advances in Learned image Compression (LIC) utilize the powerful masked image modeling (MIM) framework (e.g., MAE). However, the random masking treats all patches equally, which could lead (i) missing critical information and (ii) poor robustness, since different patches may have different contributions to image reconstruction. The principle experiment shows that the patch-level inherent feature evaluation is highly correlated with the quality of the reconstructed image. Based on this key insight, we propose a simple mask selection approach based on the patch's inherent feature. Specifically, we design plug-and-play frequency and spatial inherent feature modules to select masks, enhancing various MIMs based on the MAE model and achieving high-quality image reconstruction and strong robustness at high mask rates for data-intensive compression tasks.

关键词： image Compression masked image modeling masked Autoencoder

来源：评论

学校读者我要写书评

暂无评论

Hard semantic mask strategy for automatic facial action unit recognition with teacher-student model

引用

MULTIMEDIA SYSTEMS 2024年第4期30卷 1-12页

作者： Liang, Zichen Xia, Haiying Tan, Yumei Song, Shuxiang Guangxi Normal Univ Sch Comp Sci & Engn Guilin 541004 Peoples R China Guangxi Normal Univ Key Lab Integrated Circuits & Microsyst Guilin 541004 Peoples R China

Facial Action Coding System (FACS) is a widely used technique in affective computing, which defines a series of facial action units (AUs) corresponding to localized regions of the face. Fine-grained feature information of critical regions is crucial for accurate AU recognition. However, conventional random masking techniques used in masked image modeling (MIM) often overlook the inherent symmetry of faces and the complex interrelationships among facial muscles, leading to a lack of critical local details and poor AU recognition performance. To address these limitations, we propose a novel teacher-student model-based MIM framework called Hard Semantic Masking Strategy Teacher-Student (HSMS-TS). Specifically, we first introduce a hard semantic mask strategy in the teacher model, aims to guide the student network to focus on learning fine-grained AU-related representations. Then, the student network utilizes the attention maps from the pretrained teacher model to generate a more challenging masking method from a predefined template, increasing the learning difficulty and helping the student acquire better AU-related representations. The experimental results on two publicly available datasets, i.e., BP4D and DISFA, show the effectiveness of our proposed method with exceptional performance. Code will be publicly available at http://***/lzichen/HSMS-TS.

关键词： Facial action unit recognition masked image modeling Representation learning Deep learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：