检索结果-内蒙古大学图书馆

Separate and Conquer: Decoupling Co-occurrence via Decomposition and Representation for Weakly Supervised Semantic Segmentation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yang, Zhiwei Fu, Kexue Duan, Minghong Qu, Linhao Wang, Shuo Song, Zhijian Academy for Engineering and Technology Fudan University China Digital Medical Research Center School of Basic Medical Sciences Fudan University China Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention China Shandong Computer Science Center National Supercomputer Center in Jinan China

Attributed to the frequent coupling of co-occurring objects and the limited supervision from image-level labels, the challenging co-occurrence problem is widely present and leads to false activation of objects in weakly supervised semantic segmentation (WSSS). In this work, we devise a’Separate and Conquer’ scheme SeCo to tackle this issue from dimensions of image space and feature space. In the image space, we propose to’separate’ the co-occurring objects with image decomposition by subdividing images into patches. Importantly, we assign each patch a category tag from Class Activation Maps (CAMs), which spatially helps remove the co-context bias and guide the subsequent representation. In the feature space, we propose to’conquer’ the false activation by enhancing semantic representation with multi-granularity knowledge contrast. To this end, a dual-teacher-single-student architecture is designed and tag-guided contrast is conducted to guarantee the correctness of knowledge and further facilitate the discrepancy among co-occurring objects. We streamline the multi-staged WSSS pipeline end-to-end and tackle co-occurrence without external supervision. Extensive experiments are conducted, validating the efficiency of our method tackling co-occurrence and the superiority over previous single-staged and even multi-staged competitors on PASCAL VOC and MS COCO. Code will be available here. Copyright © 2024, The Authors. All rights reserved.

关键词： Chemical activation

PointMBF: A Multi-scale Bidirectional Fusion Network for Unsupervised RGB-D Point Cloud Registration

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Yuan, Mingzhi Fu, Kexue Li, Zhihao Meng, Yucong Wang, Manning Digital Medical Research Center School of Basic Medical Sciences Fudan University China Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention China Shandong Computer Science Center National Supercomputer Center Jinan China

Point cloud registration is a task to estimate the rigid transformation between two unaligned scans, which plays an important role in many computer vision applications. Previous learning-based works commonly focus on supervised registration, which have limitations in practice. Recently, with the advance of inexpensive RGB-D sensors, several learning-based works utilize RGB-D data to achieve unsupervised registration. However, most of existing unsupervised methods follow a cascaded design or fuse RGB-D data in a unidirectional manner, which do not fully exploit the complementary information in the RGB-D data. To leverage the complementary information more effectively, we propose a network implementing multi-scale bidirectional fusion between RGB images and point clouds generated from depth images. By bidirectionally fusing visual and geometric features in multi-scales, more distinctive deep features for correspondence estimation can be obtained, making our registration more accurate. Extensive experiments on ScanNet and 3DMatch demonstrate that our method achieves new state-of-the-art performance. Code will be released at https://***/phdymz/PointMBF. Copyright © 2023, The Authors. All rights reserved.

关键词： Surface measurement

Pos-Bert: Point Cloud One-Stage Bert Pre-Training

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Fu, Kexue Gao, Peng Liu, Shaolei Wang, Manning Digital Medical Research Center School of Basic Medical Science Fudan University China Shandong Computer Science Center China Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention China Shanghai AILab China

Recently, the pre-training paradigm combining Transformer and masked language modeling in BERT has achieved tremendous success not only in NLP, but also in images and point clouds. However, directly extending BERT from NLP to point clouds requires first training a discrete Variational AutoEncoder (dVAE) as the tokenizer, which results in a complex two-stage process, as in Point-BERT. Inspired by BERT and MoCo, we propose POS-BERT, a one-stage BERT pre-training method for point clouds. Specifically, we use the masked patch modeling (MPM) task to perform point cloud pre-training, which aims to recover masked patch information under the supervision of a tokenizer's output. Unlike Point-BERT, whose tokenizer is extra-trained and frozen, we propose a momentum tokenizer which is dynamically updated during training the Transformer. Furthermore, in order to better learn high-level semantic representation, we integrate contrastive learning into the proposed framework to maximize the class token consistency between augmented point cloud pairs. Experiments show that POS-BERT achieves the state-of-the-art performance on linear SVM classification of ModelNet40 with fixed feature extractors, and it exceeds Point-BERT by 3.5%. In addition, POS-BERT has significantly improved many downstream tasks, including fine-tuned classification, few-shot classification and part segmentation. The code and the trained model will be released soon. © 2023, The Authors. All rights reserved.

关键词： Natural language processing systems

Boosting Point-BERT by Multi-choice Tokens

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Fu, Kexue Yuan, Mingzhi Wang, Manning Digital Medical Research Center School of Basic Medical Science Fudan University Shanghai China Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention Shanghai China

Masked language modeling (MLM) has become one of the most successful self-supervised pre-training task. Inspired by its success, Point-BERT, as a pioneer work in point cloud, proposed masked point modeling (MPM) to pre-train point transformer on large scale unanotated dataset. Despite its great performance, we find the inherent difference between language and point cloud tends to cause ambiguous tokenization for point cloud. For point cloud, there doesn’t exist a gold standard for point cloud tokenization. Point-BERT use a discrete Variational AutoEncoder (dVAE) as tokenizer, but it might generate different token ids for semantically-similar patches and generate the same token ids for semantically-dissimilar patches. To tackle above problem, we propose our McP-BERT, a pre-training framework with multi-choice tokens. Specifically, we ease the previous single-choice constraint on patch token ids in Point-BERT, and provide multi-choice token ids for each patch as supervision. Moreover, we utilitze the high-level semantics learned by transformer to further refine our supervision signals. Extensive experiments on point cloud classification, few-shot classification and part segmentation tasks demonstrate the superiority of our method, e.g., the pre-trained transformer achieves 94.1% accuracy on ModelNet40, 84.28% accuracy on the hardest setting of ScanObjectNN and new state-of-the-art performance on few-shot learning. We also demonstrate that our method not only improves the performance of Point-BERT on all downstream tasks, but also incurs almost no extra computational overhead. The code will be released in https://***/fukexue/McP-BERT. Copyright © 2022, The Authors. All rights reserved.

关键词： Large dataset

MoRe: Class Patch Attention Needs Regularization for Weakly Supervised Semantic Segmentation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yang, Zhiwei Meng, Yucong Fu, Kexue Wang, Shuo Song, Zhijian Academy for Engineering and Technology Fudan University Shanghai200433 China Digital Medical Research Center School of Basic Medical Sciences Fudan University Shanghai200032 China Shanghai Key Laboratory of Medical Imaging Computing and Computer Assisted Intervention Shanghai200032 China China

Weakly Supervised Semantic Segmentation (WSSS) with image-level labels typically uses Class Activation Maps (CAM) to achieve dense predictions. Recently, Vision Transformer (ViT) has provided an alternative to generate localization maps from class-patch attention. However, due to insufficient constraints on modeling such attention, we observe that the Localization Attention Maps (LAM) often struggle with the artifact issue, i.e., patch regions with minimal semantic relevance are falsely activated by class tokens. In this work, we propose MoRe to address this issue and further explore the potential of LAM. Our findings suggest that imposing additional regularization on class-patch attention is necessary. To this end, we first view the attention as a novel directed graph and propose the Graph Category Representation module to implicitly regularize the interaction among class-patch entities. It ensures that class tokens dynamically condense the related patch information and suppress unrelated artifacts at a graph level. Second, motivated by the observation that CAM from classification weights maintains smooth localization of objects, we devise the Localization-informed Regularization module to explicitly regularize the class-patch attention. It directly mines the token relations from CAM and further supervises the consistency between class and patch tokens in a learnable manner. Extensive experiments are conducted on PASCAL VOC and MS COCO, validating that MoRe effectively addresses the artifact issue and achieves state-of-the-art performance, surpassing recent single-stage and even multi-stage methods. Code is available at https://***/zwyang6/MoRe. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantic Segmentation

Towards Arbitrary-scale Histopathology image Super-resolution: An Efficient Dual-branch Framework based on Implicit Self-texture Enhancement

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Qu, Linhao Duan, Minghong Yang, Zhiwei Wang, Manning Song, Zhijian Digital Medical Research Center School of Basic Medical Science Fudan University Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention Shanghai200032 China Academy for Engineering & Technology Fudan University Shanghai200433 China

Existing super-resolution models for pathology images can only work in fixed integer magnifications and have limited performance. Though implicit neural network-based methods have shown promising results in arbitrary-scale super-resolution of natural images, it is not effective to directly apply them in pathology images, because pathology images have special fine-grained image textures different from natural images. To address this challenge, we propose a dual-branch framework with an efficient self-texture enhancement mechanism for arbitrary-scale super-resolution of pathology images. Extensive experiments on two public datasets show that our method outperforms both existing fixed-scale and arbitrary-scale algorithms. To the best of our knowledge, this is the first work to achieve arbitrary-scale super-resolution in the field of pathology images. Codes will be available. Copyright © 2023, The Authors. All rights reserved.

关键词： Pathology

FAST: a dual-tier few-shot learning paradigm for whole slide image classification 24

学校读者我要写书评

暂无评论

FAST: a dual-tier few-shot learning paradigm for whole slide...

Proceedings of the 38th International Conference on Neural Information Processing Systems

作者： Kexue Fu Xiaoyuan Luo Linhao Qu Shuo Wang Ying Xiong Ilias Maglogiannis Longxiang Gao Manning Wang Key Laboratory of Computing Power Network and Information Security Ministry of Education Shandong Computer Science Center (National Supercomputer Center in Jinan) Qilu University of Technology (Shandong Academy of Sciences) Jinan China and Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing Shandong Fundamental Research Center for Computer Science Jinan China Digital Medical Research Center School of Basic Medical Sciences Fudan University and Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention Fudan University University of Piraeus

ISBN: (纸本)9798331314385

The expensive fine-grained annotation and data scarcity have become the primary obstacles for the widespread adoption of deep learning-based Whole Slide images (WSI) classification algorithms in clinical practice. Unlike few-shot learning methods in natural images that can leverage the labels of each image, existing few-shot WSI classification methods only utilize a small number of fine-grained labels or weakly supervised slide labels for training in order to avoid expensive fine-grained annotation. They lack sufficient mining of available WSIs, severely limiting WSI classification performance. To address the above issues, we propose a novel and efficient dual-tier few-shot learning paradigm for WSI classification, named FAST. FAST consists of a dual-level annotation strategy and a dual-branch classification framework. Firstly, to avoid expensive fine-grained annotation, we collect a very small number of WSIs at the slide level, and annotate an extremely small number of patches. Then, to fully mining the available WSIs, we use all the patches and available patch labels to build a cache branch, which utilizes the labeled patches to learn the labels of unlabeled patches and through knowledge retrieval for patch classification. In addition to the cache branch, we also construct a prior branch that includes learnable prompt vectors, using the text encoder of visual-language models for patch classification. Finally, we integrate the results from both branches to achieve WSI classification. Extensive experiments on binary and multi-class datasets demonstrate that our proposed method significantly surpasses existing few-shot classification methods and approaches the accuracy of fully supervised methods with only 0.22% annotation costs. All codes and models will be publicly available on https://***/fukexue/FAST.

关键词：

Robust Point Cloud Registration Framework Based on Deep Graph Matching(TPAMI Version)

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Fu, Kexue Luo, Jiazheng Luo, Xiaoyuan Liu, Shaolei Zhang, Chenxi Wang, Manning The Digital Medical Research Center School of Basic Medical Science Fudan University Shanghai200032 China The Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention Shanghai200032 China

3D point cloud registration is a fundamental problem in computer vision and robotics. Recently, learning-based point cloud registration methods have made great progress. However, these methods are sensitive to outliers, which lead to more incorrect correspondences. In this paper, we propose a novel deep graph matching-based framework for point cloud registration. Specifically, we first transform point clouds into graphs and extract deep features for each point. Then, we develop a module based on deep graph matching to calculate a soft correspondence matrix. By using graph matching, not only the local geometry of each point but also its structure and topology in a larger range are considered in establishing correspondences, so that more correct correspondences are found. We train the network with a loss directly defined on the correspondences, and in the test stage the soft correspondences are transformed into hard one-to-one correspondences so that registration can be performed by a correspondence-based solver. Furthermore, we introduce a transformer-based method to generate edges for graph construction, which further improves the quality of the correspondences. Extensive experiments on object-level and scene-level benchmark datasets show that the proposed method achieves state-of-the-art performance. The code is available at: https://***/fukexue/RGM. Copyright © 2022, The Authors. All rights reserved.

关键词： Topology

MoRe: Class Patch Attention Needs Regularization for Weakly Supervised Semantic Segmentation 39

学校读者我要写书评

暂无评论

MoRe: Class Patch Attention Needs Regularization for Weakly ...

39th Annual AAAI Conference on Artificial Intelligence, AAAI 2025

ISBN: (纸本)157735897X

Weakly Supervised Semantic Segmentation (WSSS) with image-level labels typically uses Class Activation Maps (CAM) to achieve dense predictions. Recently, Vision Transformer (ViT) has provided an alternative to generate localization maps from class-patch attention. However, due to insufficient constraints on modeling such attention, we observe that the Localization Attention Maps (LAM) often struggle with the artifact issue, i.e., patch regions with minimal semantic relevance are falsely activated by class tokens. In this work, we propose MoRe to address this issue and further explore the potential of LAM. Our findings suggest that imposing additional regularization on class-patch attention is necessary. To this end, we first view the attention as a novel directed graph and propose the Graph Category Representation module to implicitly regularize the interaction among class-patch entities. It ensures that class tokens dynamically condense the related patch information and suppress unrelated artifacts at a graph level. Second, motivated by the observation that CAM from classification weights maintains smooth localization of objects, we devise the Localization-informed Regularization module to explicitly regularize the class-patch attention. It directly mines the token relations from CAM and further supervises the consistency between class and patch tokens in a learnable manner. Extensive experiments on PASCAL VOC and MS COCO validate that MoRe effectively addresses the artifact issue and achieves state-of-the-art performance, surpassing recent single-stage and even multi-stage methods. Copyright © 2025, Association for the Advancement of Artificial Intelligence (***). All rights reserved.

关键词： Flow graphs