检索结果-内蒙古大学图书馆

Boosting 3D Point Cloud Registration by Transferring Multi-modality Knowledge

学校读者我要写书评

暂无评论

Boosting 3D Point Cloud Registration by Transferring Multi-m...

IEEE International Conference on Robotics and Automation (ICRA)

作者： Mingzhi Yuan Xiaoshui Huang Kexue Fu Zhihao Li Manning Wang Digital Medical Research Center School of Basic Medical Science Fudan University Shanghai China Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention Shanghai China Shanghai artificial intelligence Lab. China

The recent multi-modality models have achieved great performance in many vision tasks because the extracted features contain the multi-modality knowledge. However, most of the current registration descriptors have only concentrated on local geometric structures. This paper proposes a method to boost point cloud registration accuracy by transferring the multi-modality knowledge of pre-trained multi-modality model to a new descriptor neural network. Different to the previous multi-modality methods that requires both modalities, the proposed method only requires point clouds during inference. Specifically, we propose an ensemble descriptor neural network combining pre-trained sparse convolution branch and a new point-based convolution branch. By fine-tuning on a single modality data, the proposed method achieves new state-of-the-art results on 3DMatch and competitive accuracy on 3DLoMatch and KITTI. The code and the trained model will be released at https://***/phdymz/***.

关键词：

Pos-Bert: Point Cloud One-Stage Bert Pre-Training

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Fu, Kexue Gao, Peng Liu, Shaolei Wang, Manning Digital Medical Research Center School of Basic Medical Science Fudan University China Shandong Computer Science Center China Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention China Shanghai AILab China

Recently, the pre-training paradigm combining Transformer and masked language modeling in BERT has achieved tremendous success not only in NLP, but also in images and point clouds. However, directly extending BERT from NLP to point clouds requires first training a discrete Variational AutoEncoder (dVAE) as the tokenizer, which results in a complex two-stage process, as in Point-BERT. Inspired by BERT and MoCo, we propose POS-BERT, a one-stage BERT pre-training method for point clouds. Specifically, we use the masked patch modeling (MPM) task to perform point cloud pre-training, which aims to recover masked patch information under the supervision of a tokenizer's output. Unlike Point-BERT, whose tokenizer is extra-trained and frozen, we propose a momentum tokenizer which is dynamically updated during training the Transformer. Furthermore, in order to better learn high-level semantic representation, we integrate contrastive learning into the proposed framework to maximize the class token consistency between augmented point cloud pairs. Experiments show that POS-BERT achieves the state-of-the-art performance on linear SVM classification of ModelNet40 with fixed feature extractors, and it exceeds Point-BERT by 3.5%. In addition, POS-BERT has significantly improved many downstream tasks, including fine-tuned classification, few-shot classification and part segmentation. The code and the trained model will be released soon. © 2023, The Authors. All rights reserved.

关键词： Natural language processing systems

FAST: a dual-tier few-shot learning paradigm for whole slide image classification 24

学校读者我要写书评

暂无评论

FAST: a dual-tier few-shot learning paradigm for whole slide...

Proceedings of the 38th International Conference on Neural Information Processing Systems

作者： Kexue Fu Xiaoyuan Luo Linhao Qu Shuo Wang Ying Xiong Ilias Maglogiannis Longxiang Gao Manning Wang Key Laboratory of Computing Power Network and Information Security Ministry of Education Shandong Computer Science Center (National Supercomputer Center in Jinan) Qilu University of Technology (Shandong Academy of Sciences) Jinan China and Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing Shandong Fundamental Research Center for Computer Science Jinan China Digital Medical Research Center School of Basic Medical Sciences Fudan University and Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention Fudan University University of Piraeus

ISBN: (纸本)9798331314385

The expensive fine-grained annotation and data scarcity have become the primary obstacles for the widespread adoption of deep learning-based Whole Slide images (WSI) classification algorithms in clinical practice. Unlike few-shot learning methods in natural images that can leverage the labels of each image, existing few-shot WSI classification methods only utilize a small number of fine-grained labels or weakly supervised slide labels for training in order to avoid expensive fine-grained annotation. They lack sufficient mining of available WSIs, severely limiting WSI classification performance. To address the above issues, we propose a novel and efficient dual-tier few-shot learning paradigm for WSI classification, named FAST. FAST consists of a dual-level annotation strategy and a dual-branch classification framework. Firstly, to avoid expensive fine-grained annotation, we collect a very small number of WSIs at the slide level, and annotate an extremely small number of patches. Then, to fully mining the available WSIs, we use all the patches and available patch labels to build a cache branch, which utilizes the labeled patches to learn the labels of unlabeled patches and through knowledge retrieval for patch classification. In addition to the cache branch, we also construct a prior branch that includes learnable prompt vectors, using the text encoder of visual-language models for patch classification. Finally, we integrate the results from both branches to achieve WSI classification. Extensive experiments on binary and multi-class datasets demonstrate that our proposed method significantly surpasses existing few-shot classification methods and approaches the accuracy of fully supervised methods with only 0.22% annotation costs. All codes and models will be publicly available on https://***/fukexue/FAST.

关键词：

Tackling Ambiguity from Perspective of Uncertainty Inference and Affinity Diversification for Weakly Supervised Semantic Segmentation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yang, Zhiwei Meng, Yucong Fu, Kexue Wang, Shuo Song, Zhijian Academy for Engineering & Technology Fudan University Shanghai200433 China Digital Medical Research Center School of Basic Medical Science Fudan University China Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention Shanghai200032 China Shandong Computer Science Center China

Weakly supervised semantic segmentation (WSSS) with image-level labels intends to achieve dense tasks without laborious annotations. However, due to the ambiguous contexts and fuzzy regions, the performance of WSSS, especially the stages of generating Class Activation Maps (CAMs) and refining pseudo masks, widely suffers from ambiguity while being barely noticed by previous literature. In this work, we propose UniA, a unified single-staged WSSS framework, to efficiently tackle this issue from the perspective of uncertainty inference and affinity diversification, respectively. When activating class objects, we argue that the false activation stems from the bias to the ambiguous regions during the feature extraction. Therefore, we design a more robust feature representation with a probabilistic Gaussian distribution and introduce the uncertainty estimation to avoid the bias. A distribution loss is particularly proposed to supervise the process, which effectively captures the ambiguity and models the complex dependencies among features. When refining pseudo labels, we observe that the affinity from the prevailing refinement methods intends to be similar among ambiguities. To this end, an affinity diversification module is proposed to promote diversity among semantics. A mutual complementing refinement is proposed to initially rectify the ambiguous affinity with multiple inferred pseudo labels. More importantly, a contrastive affinity loss is further designed to diversify the relations among unrelated semantics, which reliably propagates the diversity into the whole feature representations and helps generate better pseudo masks. Extensive experiments are conducted on PASCAL VOC, MS COCO, and medical ACDC datasets, which validate the efficiency of UniA tackling ambiguity and the superiority over recent single-staged or even most multi-staged competitors. Codes will be publicly available here. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantic Segmentation

Robust and faster zeroth-order minimax optimization: complexity and applications 24

学校读者我要写书评

暂无评论

Robust and faster zeroth-order minimax optimization: complex...

Proceedings of the 38th International Conference on Neural Information Processing Systems

作者： Weixin An Yuanyuan Liu Fanhua Shang Hongying Liu Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education School of Artificial Intelligence Xidian University China College of Intelligence and Computing Tianjin University China Medical School Tianjin University China and Peng Cheng Lab Shenzhen China

ISBN: (纸本)9798331314385

Many zeroth-order (ZO) optimization algorithms have been developed to solve nonconvex minimax problems in machine learning and computer vision areas. However, existing ZO minimax algorithms have high complexity and rely on some strict restrictive conditions for ZO estimations. To address these issues, we design a new unified ZO gradient descent extragradient ascent (ZO-GDEGA) algorithm, which reduces the overall complexity to O(dε-6) to find an ε-stationary point of the function ψ for nonconvex-concave (NC-C) problems, where d is the variable dimension. To the best of our knowledge, ZO-GDEGA is the first ZO algorithm with complexity guarantees to solve stochastic NC-C problems. Moreover, ZO-GDEGA requires weaker conditions on the ZO estimations and achieves more robust theoretical results. As a by-product, ZO-GDEGA has advantages on the condition number for the NC-strongly concave case. Experimentally, ZO-GDEGA can generate more effective poisoning attack data with an average accuracy reduction of 5%. The improved AUC performance also verifies the robustness of gradient estimations.

关键词：

Boosting 3D Point Cloud Registration by Transferring Multi-modality Knowledge

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Yuan, Mingzhi Huang, Xiaoshui Fu, Kexue Li, Zhihao Wang, Manning The Digital Medical Research Center School of Basic Medical Science Fudan University Shanghai200032 China The Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention Shanghai200032 China The Shanghai artificial intelligence Lab China

关键词： Convolution

Bi-directional Weakly Supervised Knowledge Distillation for Whole Slide image Classification

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Qu, Linhao Luo, Xiaoyuan Wang, Manning Song, Zhijian Digital Medical Research Center School of Basic Medical Science Fudan University China Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention China

Computer-aided pathology diagnosis based on the classification of Whole Slide image (WSI) plays an important role in clinical practice, and it is often formulated as a weakly-supervised Multiple Instance Learning (MIL) problem. Existing methods solve this problem from either a bag classification or an instance classification perspective. In this paper, we propose an end-to-end weakly supervised knowledge distillation framework (WENO) for WSI classification, which integrates a bag classifier and an instance classifier in a knowledge distillation framework to mutually improve the performance of both classifiers. Specifically, an attention-based bag classifier is used as the teacher network, which is trained with weak bag labels, and an instance classifier is used as the student network, which is trained using the normalized attention scores obtained from the teacher network as soft pseudo labels for the instances in positive bags. An instance feature extractor is shared between the teacher and the student to further enhance the knowledge exchange between them. In addition, we propose a hard positive instance mining strategy based on the output of the student network to force the teacher network to keep mining hard positive instances. WENO is a plug-and-play framework that can be easily applied to any existing attention-based bag classification methods. Extensive experiments on five datasets demonstrate the efficiency of WENO. Code is available at https://***/miccaiif/WENO. Copyright © 2022, The Authors. All rights reserved.

关键词： Distillation

Towards Arbitrary-scale Histopathology image Super-resolution: An Efficient Dual-branch Framework based on Implicit Self-texture Enhancement

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Qu, Linhao Duan, Minghong Yang, Zhiwei Wang, Manning Song, Zhijian Digital Medical Research Center School of Basic Medical Science Fudan University Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention Shanghai200032 China Academy for Engineering & Technology Fudan University Shanghai200433 China

Existing super-resolution models for pathology images can only work in fixed integer magnifications and have limited performance. Though implicit neural network-based methods have shown promising results in arbitrary-scale super-resolution of natural images, it is not effective to directly apply them in pathology images, because pathology images have special fine-grained image textures different from natural images. To address this challenge, we propose a dual-branch framework with an efficient self-texture enhancement mechanism for arbitrary-scale super-resolution of pathology images. Extensive experiments on two public datasets show that our method outperforms both existing fixed-scale and arbitrary-scale algorithms. To the best of our knowledge, this is the first work to achieve arbitrary-scale super-resolution in the field of pathology images. Codes will be available. Copyright © 2023, The Authors. All rights reserved.

关键词： Pathology

DermSynth3D: Synthesis of in-the-wild Annotated Dermatology images

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Sinha, Ashish Kawahara, Jeremy Pakzad, Arezou Abhishek, Kumar Ruthven, Matthieu Ghorbel, Enjie Kacem, Anis Aouada, Djamila Hamarneh, Ghassan Medical Image Analysis Lab School of Computing Science Simon Fraser University Canada University of Luxembourg Luxembourg Cristal Laboratory National School of Computer Sciences University of Manouba 2010 Tunisia

In recent years, deep learning (DL) has shown great potential in the field of dermatological image analysis. However, existing datasets in this domain have significant limitations, including a small number of image samples, limited disease conditions, insufficient annotations, and non-standardized image acquisitions. To address these shortcomings, we propose a novel framework called DermSynth3D. DermSynth3D blends skin disease patterns onto 3D textured meshes of human subjects using a differentiable renderer and generates 2D images from various camera viewpoints under chosen lighting conditions in diverse background scenes. Our method adheres to top-down rules that constrain the blending and rendering process to create 2D images with skin conditions that mimic in-the-wild acquisitions, ensuring more meaningful results. The framework generates photo-realistic 2D dermoscopy images and the corresponding dense annotations for semantic segmentation of the skin, skin conditions, body parts, bounding boxes around lesions, depth maps, and other 3D scene parameters, such as camera position and lighting conditions. DermSynth3D allows for the creation of custom datasets for various dermatology tasks. We demonstrate the effectiveness of data generated using DermSynth3D by training DL models on synthetic data and evaluating them on various dermatology tasks using real 2D dermatological images. We make our code publicly available at https://***/sfu-mial/DermSynth3D. © 2023, CC BY-NC-SA.

关键词： Cameras