检索结果-内蒙古大学图书馆

POS-BERT: Point Cloud One-Stage BERT Pre-Training

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Fu, Kexue Gao, Peng Liu, Shaolei Zhang, Renrui Qiao, Yu Wang, Manning Digital Medical Research Center School of Basic Medical Sciences Fudan University China Shanghai AI Lab China Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention China

Recently, the pre-training paradigm combining Transformer and masked language modeling has achieved tremendous success in NLP, images, and point clouds, such as BERT. However, directly extending BERT from NLP to point clouds requires training a fixed discrete Variational AutoEncoder (dVAE) before pre-training, which results in a complex two-stage method called Point-BERT. Inspired by BERT and MoCo, we propose POS-BERT, a one-stage BERT pre-training method for point clouds. Specifically, we use the mask patch modeling (MPM) task to perform point cloud pre-training, which aims to recover masked patches information under the supervision of the corresponding tokenizer output. Unlike Point-BERT, its tokenizer is extra-trained and frozen. We propose to use the dynamically updated momentum encoder as the tokenizer, which is updated and outputs the dynamic supervision signal along with the training process. Further, in order to learn high-level semantic representation, we combine contrastive learning to maximize the class token consistency between different transformation point clouds. Extensive experiments have demonstrated that POS-BERT can extract high-quality pre-training features and promote downstream tasks to improve performance. Using the pretraining model without any fine-tuning to extract features and train linear SVM on ModelNet40, POS-BERT achieves the state-of-the-art classification accuracy, which exceeds Point-BERT by 3.5%. In addition, our approach has significantly improved many downstream tasks, such as fine-tuned classification, few-shot classification, part segmentation. The code and trained-models will be available at: https://***/fukexue/POS-BERT. Copyright © 2022, The Authors. All rights reserved.

关键词： Natural language processing systems

A Five-Channel Weighted Real-Time Algorithm for High-Density Electrodes Spike Sorting

学校读者我要写书评

暂无评论

A Five-Channel Weighted Real-Time Algorithm for High-Density...

Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)

作者： Jiaxin He Chongyuan Ren Yu Ma Yizhou Jiang Yajie Qin School of Information Science and Technology Fudan University Shanghai China Key Laboratory of Medical Imaging Computing and Computer Assisted Intervention (MICCAI) of Shanghai Shanghai China Shanghai Engineering Research Center of Assistive Devices Shanghai China

With the application of high-density neural probes, a neuron can be detected by multiple adjacent probes, and the traditional single-channel spike sorting is no longer suitable. In this paper, we propose a five-channel weighted real-time spike sorting algorithm based on template-matching to process neural signals recorded by high-density probes. This work uses the signals of the center channel and the adjacent four channels to form a five-channel template by weighting, and employs a modified OSort algorithm with unsupervised learning to update the template. We implemented automatic online spike sorting, and tested it with both ground truth recordings and simulated datasets. The experiments show that our algorithm utilizing the information of adjacent channels has a higher sorting accuracy than traditional single-channel spike sorting. The average sorting accuracy reaches 89%, compared to 78% for single-channel.

关键词：

TransFuse: A Unified Transformer-based image Fusion Framework using Self-supervised Learning

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Qu, Linhao Liu, Shaolei Wang, Manning Li, Shiman Yin, Siqi Qiao, Qin Song, Zhijian Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention Digital Medical Research Center School of Basic Medical Science Fudan University Shanghai200032 China

image fusion is a technique to integrate information from multiple source images with complementary information to improve the richness of a single image. Due to insufficient task-specific training data and corresponding ground truth, most existing end-to-end image fusion methods easily fall into overfitting or tedious parameter optimization processes. Two-stage methods avoid the need of large amount of task-specific training data by training encoder-decoder network on large natural image datasets and utilizing the extracted features for fusion, but the domain gap between natural images and different fusion tasks results in limited performance. In this study, we design a novel encoder-decoder based image fusion framework and propose a destruction-reconstruction based self-supervised training scheme to encourage the network to learn task-specific features. Specifically, we propose three destruction-reconstruction self-supervised auxiliary tasks for multi-modal image fusion, multi-exposure image fusion and multi-focus image fusion based on pixel intensity non-linear transformation, brightness transformation and noise transformation, respectively. In order to encourage different fusion tasks to promote each other and increase the generalizability of the trained network, we integrate the three self-supervised auxiliary tasks by randomly choosing one of them to destroy a natural image in model training. In addition, we design a new encoder that combines CNN and Transformer for feature extraction, so that the trained model can exploit both local and global information. Extensive experiments on multi-modal image fusion, multi-exposure image fusion and multi-focus image fusion tasks demonstrate that our proposed method achieves the state-of-the-art performance in both subjective and objective evaluations. The code will be publicly available soon. Copyright © 2022, The Authors. All rights reserved.

关键词： Deep learning

DGMIL: Distribution Guided Multiple Instance Learning for Whole Slide image Classification

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Qu, Linhao Luo, Xiaoyuan Liu, Shaolei Wang, Manning Song, Zhijian Digital Medical Research Center School of Basic Medical Science Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention Fudan University Shanghai200032 China

Multiple Instance Learning (MIL) is widely used in analyzing histopathological Whole Slide images (WSIs). However, existing MIL methods do not explicitly model the data distribution, and instead they only learn a bag-level or instance-level decision boundary discriminatively by training a classifier. In this paper, we propose DGMIL: a feature distribution guided deep MIL framework for WSI classification and positive patch localization. Instead of designing complex discriminative network architectures, we reveal that the inherent feature distribution of histopathological image data can serve as a very effective guide for instance classification. We propose a cluster-conditioned feature distribution modeling method and a pseudo label-based iterative feature space refinement strategy so that in the final feature space the positive and negative instances can be easily separated. Experiments on the CAMELYON16 dataset and the TCGA Lung Cancer dataset show that our method achieves new SOTA for both global classification and positive patch localization tasks. Copyright © 2022, The Authors. All rights reserved.

关键词： image classification

Rethinking Multi-Exposure image Fusion with Extreme and Diverse Exposure Levels: A Robust Framework Based on Fourier Transform and Contrastive Learning

学校读者我要写书评

暂无评论

SSRN

SSRN 2022年

作者： Qu, Linhao Liu, Shaolei Wang, Manning Song, Zhijian Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention Digital Medical Research Center School of Basic Medical Science Fudan University Shanghai200032 China

Multi-exposure image fusion (MEF) is an important technique for generating high dynamic range images. However, most existing MEF studies focus on fusing a moderately over-exposed image and a moderately under-exposed image, and they are not robust in fusing images with extreme and diverse exposure levels. In this paper, we propose a robust MEF framework based on Fourier transform and contrastive learning. Specifically, we develop a Fourier transform-based pixel intensity transfer strategy to synthesize images with diverse exposure levels from normally exposed natural images and train an encoder-decoder network to reconstruct the original natural image. In this way, the encoder and decoder can learn to extract features from images with diverse exposure levels and generate fused images with normal exposure. We propose a contrastive regularization loss to further enhance the capability of the network in recovering normal exposure levels. In addition, we construct an extreme MEF benchmark dataset and a random MEF benchmark dataset for a more comprehensive evaluation of MEF algorithms. We extensively compare our method with nine competitive traditional and deep learning-based MEF algorithms on three benchmark datasets, and our method outperforms the other methods in both subjective visual effects and objective evaluation metrics. Our code, datasets and all fused images will be released. © 2022, The Authors. All rights reserved.

关键词： Fourier transforms

Towards Label-efficient Automatic Diagnosis and Analysis: A Comprehensive Survey of Advanced Deep Learning-based Weakly-supervised, Semi-supervised and Self-supervised Techniques in Histopathological image Analysis

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Qu, Linhao Liu, Siyu Liu, Xiaoyu Wang, Manning Song, Zhijian Digital Medical Research Center School of Basic Medical Science Fudan University Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention Shanghai200032 China

Histopathological images contain abundant phenotypic information and pathological patterns, which are the gold standards for disease diagnosis and essential for the prediction of patient prognosis and treatment outcome. In recent years, computer-automated analysis techniques for histopathological images have been urgently required in clinical practice, and deep learning methods represented by convolutional neural networks have gradually become the mainstream in the field of digital pathology. However, obtaining large numbers of fine-grained annotated data in this field is a very expensive and difficult task, which hinders the further development of traditional supervised algorithms based on large numbers of annotated data. More recent studies have started to liberate from the traditional supervised paradigm, and the most representative ones are the studies on weakly supervised learning paradigm based on weak annotation, semi-supervised learning paradigm based on limited annotation, and self-supervised learning paradigm based on pathological image representation learning. These new methods have led a new wave of automatic pathological image diagnosis and analysis targeted at annotation efficiency. With a survey of over 130 papers, we present a comprehensive and systematic review of the latest studies on weakly supervised learning, semi-supervised learning, and self-supervised learning in the field of computational pathology from both technical and methodological perspectives. Finally, we present the key challenges and future trends for these techniques. Copyright © 2022, The Authors. All rights reserved.

关键词： Surveys

Transfuse: A Unified Transformer-Based image Fusion Framework Using Self-Supervised Learning

学校读者我要写书评

暂无评论

SSRN

SSRN 2022年

关键词： Deep learning

Bi-directional Weakly Supervised Knowledge Distillation for Whole Slide image Classification

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Qu, Linhao Luo, Xiaoyuan Wang, Manning Song, Zhijian Digital Medical Research Center School of Basic Medical Science Fudan University China Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention China

computer-aided pathology diagnosis based on the classification of Whole Slide image (WSI) plays an important role in clinical practice, and it is often formulated as a weakly-supervised Multiple Instance Learning (MIL) problem. Existing methods solve this problem from either a bag classification or an instance classification perspective. In this paper, we propose an end-to-end weakly supervised knowledge distillation framework (WENO) for WSI classification, which integrates a bag classifier and an instance classifier in a knowledge distillation framework to mutually improve the performance of both classifiers. Specifically, an attention-based bag classifier is used as the teacher network, which is trained with weak bag labels, and an instance classifier is used as the student network, which is trained using the normalized attention scores obtained from the teacher network as soft pseudo labels for the instances in positive bags. An instance feature extractor is shared between the teacher and the student to further enhance the knowledge exchange between them. In addition, we propose a hard positive instance mining strategy based on the output of the student network to force the teacher network to keep mining hard positive instances. WENO is a plug-and-play framework that can be easily applied to any existing attention-based bag classification methods. Extensive experiments on five datasets demonstrate the efficiency of WENO. Code is available at https://***/miccaiif/WENO. Copyright © 2022, The Authors. All rights reserved.

关键词： Distillation

Reducing Domain Gap in Frequency and Spatial domain for Cross-modality Domain Adaptation on medical image Segmentation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Liu, Shaolei Yin, Siqi Qu, Linhao Wang, Manning Digital Medical Research Center School of Basic Medical Science Fudan University Shanghai200032 China Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention China

Unsupervised domain adaptation (UDA) aims to learn a model trained on source domain and performs well on unlabeled target domain. In medical image segmentation field, most existing UDA methods depend on adversarial learning to address the domain gap between different image modalities, which is ineffective due to its complicated training process. In this paper, we propose a simple yet effective UDA method based on frequency and spatial domain transfer under multi-teacher distillation framework. In the frequency domain, we first introduce non-subsampled contourlet transform for identifying domain-invariant and domain-variant frequency components (DIFs and DVFs), and then keep the DIFs unchanged while replacing the DVFs of the source domain images with that of the target domain images to narrow the domain gap. In the spatial domain, we propose a batch momentum update-based histogram matching strategy to reduce the domain-variant image style bias. Experiments on two cross-modality medical image segmentation datasets (cardiac, abdominal) show that our proposed method achieves superior performance compared to state-of-the-art methods. Copyright © 2022, The Authors. All rights reserved.

关键词： Distillation