检索结果-内蒙古大学图书馆

Rethinking Multiple Instance Learning: Developing an Instance-Level Classifier via Weakly-Supervised Self-Training

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ma, Yingfan Luo, Xiaoyuan Yuan, Mingzhi Chen, Xinrong Wang, Manning The Digital Medical Research Center School of Basic Medical Science Fudan University Shanghai200032 China The Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention Shanghai200032 China

Multiple instance learning (MIL) problem is currently solved from either bag-classification or instance-classification perspective, both of which ignore important information contained in some instances and result in limited performance. For example, existing methods often face difficulty in learning hard positive instances. In this paper, we formulate MIL as a semi-supervised instance classification problem, so that all the labeled and unlabeled instances can be fully utilized to train a better classifier. The difficulty in this formulation is that all the labeled instances are negative in MIL, and traditional self-training techniques used in semi-supervised learning tend to degenerate in generating pseudo labels for the unlabeled instances in this scenario. To resolve this problem, we propose a weakly-supervised self-training method, in which we utilize the positive bag labels to construct a global constraint and a local constraint on the pseudo labels to prevent them from degenerating and force the classifier to learn hard positive instances. It is worth noting that easy positive instances are instances are far from the decision boundary in the classification process, while hard positive instances are those close to the decision boundary. Through iterative optimization, the pseudo labels can gradually approach the true labels. Extensive experiments on two MNIST synthetic datasets, five traditional MIL benchmark datasets and two histopathology whole slide image datasets show that our method achieved new SOTA performance on all of them. The code will be publicly available. Impact Statement-MIL is a primary research field within weakly supervised learning and has numerous applications, such as histopathology analysis, video anomaly detection, and drug design. However, current mainstream methods in MIL predominantly rely on bag labels to train the network, consequently overlooking the potential of instance-level supervision information. In our paper, we highlight the

关键词： Self-supervised learning

FANCL: Feature-Guided Attention Network with Curriculum Learning for Brain Metastases Segmentation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Liu, Zijiang Liu, Xiaoyu Qu, Linhao Shi, Yonghong Digital Medical Research Center School of Basic Medical Science Fudan University Shanghai200032 China Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention Shanghai200032 China

Accurate segmentation of brain metastases (BMs) in MR image is crucial for the diagnosis and followup of patients. Methods based on deep convolutional neural networks (CNNs) have achieved high segmentation performance. However, due to the loss of critical feature information caused by convolutional and pooling operations, CNNs still face great challenges in small BMs segmentation. Besides, BMs are irregular and easily confused with healthy tissues, which makes it difficult for the model to effectively learn tumor structure during training. To address these issues, this paper proposes a novel model called feature-guided attention network with curriculum learning (FANCL). Based on CNNs, FANCL utilizes the input image and its feature to establish the intrinsic connections between metastases of different sizes, which can effectively compensate for the loss of high-level feature from small tumors with the information of large tumors. Furthermore, FANCL applies the voxel-level curriculum learning strategy to help the model gradually learn the structure and details of BMs. And baseline models of varying depths are employed as curriculum-mining networks for organizing the curriculum progression. The evaluation results on the BraTS-METS 2023 dataset indicate that FANCL significantly improves the segmentation performance, confirming the effectiveness of our method. Copyright © 2024, The Authors. All rights reserved.

关键词： Curricula

Ddfp: Data-Dependent Frequency Prompt for Source Free Domain Adaptation of medical image Segmentation

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Yin, Siqi Liu, Shaolei Wang, Manning Digital Medical Research Center School of Basic Medical Science Fudan University Shanghai200032 China Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention Shanghai200032 China

Domain adaptation aims to address model performance degradation problem under domain gap. In the typical setting of unsupervised domain adaptation, labeled data from source domain and unlabeled data from target domain are used to train a target model. However, access to source domain medical data can be restricted due to privacy policies, in which scenario, the research attention has shifted to source free domain adaptation (SFDA) which only requires a trained source domain model and unlabeled target domain data for adaptation. Existing SFDA methods often utilize domain-dependent image style translation and self-supervision techniques to reduce domain gap and train the target domain model. However, the quality of domain-dependent style translated images and pseudo labels still leaves room for improvement. Moreover, training the entire model during adaptation is suboptimal under limited supervision. In this paper, we propose a novel SFDA framework to overcome these limitations. Specifically, to effectively mitigate the impact of domain gap in the initial training phase, we introduce pre-adaptation to derive a pre-adapted model for target model initialization and generate quality enhanced pseudo labels without introducing additional parameters. Furthermore, we propose a data-dependent frequency prompt to better translate target domain images into a source-like style individually. Subsequently, the style-related layers fine-tuning strategy tailored for SFDA is used to train the target model using the prompted target domain images and pseudo labels. Extensive experiments on cross-modality abdominal and cardiac SFDA segmentation tasks demonstrated that our proposed method outperforms the state-of-the-art baselines. Our code is available online. © 2024, The Authors. All rights reserved.

关键词： Semantic Segmentation

Deep Mutual Learning among Partially Labeled Datasets for Multi-Organ Segmentation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Liu, Xiaoyu Qu, Linhao Xie, Ziyue Shi, Yonghong Song, Zhijian Digital Medical Research Center School of Basic Medical Science Fudan University Shanghai200032 China Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention Shanghai200032 China

The task of labeling multiple organs for segmentation is a complex and time-consuming process, resulting in a scarcity of comprehensively labeled multi-organ datasets while the emergence of numerous partially labeled datasets. Current methods are inadequate in effectively utilizing the supervised information available from these datasets, thereby impeding the progress in improving the segmentation accuracy. This paper proposes a two-stage multi-organ segmentation method based on mutual learning, aiming to improve multi-organ segmentation performance by complementing information among partially labeled datasets. In the first stage, each partial-organ segmentation model utilizes the non-overlapping organ labels from different datasets and the distinct organ features extracted by different models, introducing additional mutual difference learning to generate higher quality pseudo labels for unlabeled organs. In the second stage, each full-organ segmentation model is supervised by fully labeled datasets with pseudo labels and leverages true labels from other datasets, while dynamically sharing accurate features across different models, introducing additional mutual similarity learning to enhance multi-organ segmentation performance. Extensive experiments were conducted on nine datasets that included the head and neck, chest, abdomen, and pelvis. The results indicate that our method has achieved SOTA performance in segmentation tasks that rely on partial labels, and the ablation studies have thoroughly confirmed the efficacy of the mutual learning mechanism. © 2024, CC BY.

关键词： Deep learning

Dual Attention Poser: Dual Path Body Tracking Based on Attention

学校读者我要写书评

暂无评论

Dual Attention Poser: Dual Path Body Tracking Based on Atten...

IEEE computer society Conference on computer Vision and Pattern Recognition Workshops (CVPRW)

作者： Xinhan Di Xiaokun Dai Xinkang Zhang Xinrong Chen Deepearthgo Academy for Engineering&Technology Fudan Universiry Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention Fudan University

Currently, mixed reality head-mounted displays tracking the full body of users is an important human-computer interaction mode through the pose of the head and the hands. Unfortunately, users’ virtual representation and experience is limited due to high reconstruction error when simple transformer network architecture is applied. In this paper, we present a novel model, named Dual Attention Poser, which can learn the whole body reconstruction at a high accuracy. The proposed model consists of three key modules. Among them, dual-path attention encoder is designed to extract feature of the sparse signals. Cross attention mixer module enable the fusion of representation in the double path. Attention-gated-mlp decoder is applied to decode the hidden feature from the sparse input through attention gate. Test results on the AMASS dataset show that Dual Attention Poser can reduce the error by up to 18.2% in comparison with the state-of-the-art results.

关键词：

The Rise of AI Language Pathologists: Exploring Two-level Prompt Learning for Few-shot Weakly-supervised Whole Slide image Classification

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Qu, Linhao Luo, Xiaoyuan Fu, Kexue Wang, Manning Song, Zhijian Digital Medical Research Center School of Basic Medical Science Fudan University China Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention China

This paper introduces the novel concept of few-shot weakly supervised learning for pathology Whole Slide image (WSI) classification, denoted as FSWC. A solution is proposed based on prompt learning and the utilization of a large language model, GPT-4. Since a WSI is too large and needs to be divided into patches for processing, WSI classification is commonly approached as a Multiple Instance Learning (MIL) problem. In this context, each WSI is considered a bag, and the obtained patches are treated as instances. The objective of FSWC is to classify both bags and instances with only a limited number of labeled bags. Unlike conventional few-shot learning problems, FSWC poses additional challenges due to its weak bag labels within the MIL framework. Drawing inspiration from the recent achievements of vision-language models (V-L models) in downstream few-shot classification tasks, we propose a two-level prompt learning MIL framework tailored for pathology, incorporating language prior knowledge. Specifically, we leverage CLIP to extract instance features for each patch, and introduce a prompt-guided pooling strategy to aggregate these instance features into a bag feature. Subsequently, we employ a small number of labeled bags to facilitate few-shot prompt learning based on the bag features. Our approach incorporates the utilization of GPT-4 in a question- and-answer mode to obtain language prior knowledge at both the instance and bag levels, which are then integrated into the instance and bag level language prompts. Additionally, a learnable component of the language prompts is trained using the available few-shot labeled data. We conduct extensive experiments on three real WSI datasets encompassing breast cancer, lung cancer, and cervical cancer, demonstrating the notable performance of the proposed method in bag and instance classification. Codes will be available at https://***/miccaiif/TOP. Copyright © 2023, The Authors. All rights reserved.

关键词： Pathology

SP3 : Superpixel-propagated pseudo-label learning for weakly semi-supervised medical image segmentation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Shiman Zhao, Jiayue Liu, Shaolei Dai, Xiaokun Zhang, Chenxi Song, Zhijian The School of Basic Medical Science Fudan University 200433 China Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention 200032 China The Academy for Engineering and Technology Fudan University 200433 China

Deep learning-based medical image segmentation helps assist diagnosis and accelerate the treatment process while the model training usually requires large-scale dense annotation datasets. Weakly semi-supervised medical image segmentation is an essential application because it only requires a small amount of scribbles and a large number of unlabeled data to train the model, which greatly reduces the clinician’s effort to fully annotate images. To handle the inadequate supervisory information challenge in weakly semi-supervised segmentation (WSSS), a SuperPixel-Propagated Pseudo-label (SP3) learning method is proposed, using the structural information contained in superpixel for supplemental information. Specifically, the annotation of scribbles is propagated to superpixels and thus obtains a dense annotation for supervised training. Since the quality of pseudo-labels is limited by the low-quality annotation, the beneficial superpixels selected by dynamic thresholding are used to refine pseudo-labels. Furthermore, aiming to alleviate the negative impact of noise in pseudo-label, superpixel-level uncertainty is incorporated to guide the pseudo-label supervision for stable learning. Our method achieves state-of-the-art performance on both tumor and organ segmentation datasets under the WSSS setting, using only 3% of the annotation workload compared to fully supervised methods and attaining approximately 80% Dice score. Additionally, our method outperforms eight weakly and semi-supervised methods under both weakly supervised and semi-supervised settings. Results of extensive experiments validate the effectiveness and annotation efficiency of our weakly semi-supervised segmentation, which can assist clinicians in achieving automated segmentation for organs or tumors quickly and ultimately benefit patients. © 2024, CC BY.

关键词： Superpixels

Fusionmlp: A Mlp-Based Unified image Fusion Framework

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Liu, Shaolei Qu, Linhao Wang, Manning Song, Zhijian Digital Medical Research Center School of Basic Medical Science Fudan University Shanghai200032 China Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention China

Due to the powerful feature representation capacity, deep learning-based image fusion methods have improved the fusion results for better information integration. However, some inherent limitations in convolutional neural networks (CNN) and Transformer networks impede further performance improvement. Specifically, CNN-based methods have difficulties in mining global context information;Transformer-based methods are unable to adapt to different image sizes and the computational complexity grows exponentially with the image size. To overcome these problems, we propose the first MLPbased multi-scene image fusion framework, denoted as FusionMLP. In addition, we propose a novel loss function combining the spatial and frequency domain information for model training. Extensive experiments show that FusionMLP not only outperform SOTA multi-scene image fusion methods, but also outperform SOTA fusion algorithms for specific scenes. Codes will be released at https://***/slliuEric/FusionMLP. © 2023, The Authors. All rights reserved.

关键词： image fusion

Separate and Conquer: Decoupling Co-occurrence via Decomposition and Representation for Weakly Supervised Semantic Segmentation

学校读者我要写书评

暂无评论

Separate and Conquer: Decoupling Co-occurrence via Decomposi...

Conference on computer Vision and Pattern Recognition (CVPR)

作者： Zhiwei Yang Kexue Fu Minghong Duan Linhao Qu Shuo Wang Zhijian Song Academy for Engineering and Technology Fudan University Digital Medical Research Center School of Basic Medical Sciences Fudan University Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention Shandong Computer Science Center (National Supercomputer Center in Jinan)

ISBN: (数字)9798350353006

ISBN: (纸本)9798350353013

Weakly supervised semantic segmentation (WSSS) with image-level labels aims to achieve segmentation tasks with-out dense annotations. However, attributed to the frequent coupling of co-occurring objects and the limited supervision from image-level labels, the challenging co-occurrence problem is widely present and leads to false activation of objects in WSSS. In this work, we devise a ‘Separate and Conquer’ scheme SeCo to tackle this issue from di-mensions of image space and feature space. In the im-age space, we propose to ‘separate’ the co-occurring ob-jects with image decomposition by subdividing images into patches. Importantly, we assign each patch a category tag from Class Activation Maps (CAMs), which spatially helps remove the co-context bias and guide the subsequent rep-resentation. In the feature space, we propose to ‘conquer’ the false activation by enhancing semantic representation with multi-granularity knowledge contrast. To this end, a dual-teacher-single-student architecture is designed and tag-guided contrast is conducted, which guarantee the cor-rectness of knowledge and further facilitate the discrepancy among co-contexts. We streamline the multi-staged WSSS pipeline end-to-end and tackle this issue without external supervision. Extensive experiments are conducted, validating the efficiency of our method and the superiority over previous single-staged and even multi-staged competitors on PASCAL VOC and MS COCO. Code is available here.

关键词： Couplings computer vision Codes Semantic segmentation Semantics Pipelines Noise reduction