检索结果-内蒙古大学图书馆

Lawin Transformer: Improving Semantic Segmentation Transformer with Multi-Scale Representations via Large Window Attention

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yan, Haotian Zhang, Chuang Wu, Ming Pattern Recognition and Intelligent System Lab Beijing University of Posts and Telecommunications China

Multi-scale representations are crucial for semantic segmentation. The community has witnessed the flourish of semantic segmentation convolutional neural networks (CNN) exploiting multi-scale contextual information. Motivated by that the vision transformer (ViT) is powerful in image classification, some semantic segmentation ViTs are recently proposed, most of them attaining impressive results but at a cost of computational economy. In this paper, we succeed in introducing multi-scale representations into semantic segmentation ViT via window attention mechanism and further improves the performance and efficiency. To this end, we introduce large window attention which allows the local window to query a larger area of context window at only a little computation overhead. By regulating the ratio of the context area to the query area, we enable the large window attention to capture the contextual information at multiple scales. Moreover, the framework of spatial pyramid pooling is adopted to collaborate with the large window attention, which presents a novel decoder named large window attention spatial pyramid pooling (LawinASPP) for semantic segmentation ViT. Our resulting ViT, Lawin Transformer, is composed of an efficient hierachical vision transformer (HVT) as encoder and a LawinASPP as decoder. The empirical results demonstrate that Lawin Transformer offers an improved efficiency compared to the existing method. Lawin Transformer further sets new state-of-the-art performance on Cityscapes (84.4% mIoU), ADE20K (56.2% mIoU) and COCO-Stuff datasets. The code will be released at https://***/yan-hao-tian/lawin. Copyright © 2022, The Authors. All rights reserved.

关键词： Semantic Segmentation

Cross-Layer Feature based Multi-Granularity Visual Classification

学校读者我要写书评

暂无评论

Cross-Layer Feature based Multi-Granularity Visual Classific...

IEEE Visual Communications and Image Processing (VCIP)

作者： Junhan Chen Dongliang Chang Jiyang Xie Ruoyi Du Zhanyu Ma Pattern Recognition and Intelligent System Lab. Beijing University of Posts and Telecommunications Beijing China

ISBN: (纸本)9781665475938

In contrast to traditional fine-grained visual clas-sification, multi-granularity visual classification is no longer limited to identifying the different sub-classes belonging to the same super-class (e.g., bird species, cars, and aircraft models). Instead, it gives a sequence of labels from coarse to fine (e.g., Passeriformes → Corvidae → Fish Crow), which is more convenient in practice. The key to solving this task is how to use the relationships between the different levels of labels to learn feature representations that contain different levels of granularity. Interestingly, the feature pyramid structure naturally implies different granularity of feature representation, with the shallow layers representing coarse-grained features and the deep layers representing fine-grained features. Therefore, in this paper, we exploit this property of the feature pyramid structure to decouple features and obtain feature representations corre-sponding to different granularities. Specifically, we use shallow features for coarse-grained classification and deep features for fine-grained classification. In addition, to enable fine-grained features to enhance the coarse-grained classification, we propose a feature reinforcement module based on the feature pyramid structure, where deep features are first upsampled and then combined with shallow features to make decisions. Experimental results on three widely used fine-grained image classification datasets such as CUB-200-2011, Stanford Cars, and FGVC-Aircraft validate the method's effectiveness. Code available at https://***/PRIS-CV/CGVC.

关键词： Visualization Cross layer design Codes Visual communication Image processing Fish Birds

GeoPix: Multi-Modal Large Language Model for Pixel-level Image Understanding in Remote Sensing

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Ou, Ruizhe Hu, Yuan Zhang, Fan Chen, Jiaxin Liu, Yu Pattern Recognition and Intelligent System Lab School of Artificial Intelligence Beijing University of Posts and Telecommunications Beijing100876 China Institute of Remote Sensing and Geographic Information Systems School of Earth and Space Sciences Peking University Beijing100871 China Peking University Ordos Research Institute of Energy Ordos017000 China

Multi-modal large language models (MLLMs) have achieved remarkable success in image- and region-level remote sensing (RS) image understanding tasks, such as image captioning, visual question answering, and visual grounding. However, existing RS MLLMs lack the pixel-level dialogue capability, which involves responding to user instructions with segmentation masks for specific instances. In this paper, we propose GeoPix, a RS MLLM that extends image understanding capabilities to the pixel level. This is achieved by equipping the MLLM with a mask predictor, which transforms visual features from the vision encoder into masks conditioned on the LLM’s segmentation token embeddings. To facilitate the segmentation of multi-scale objects in RS imagery, a class-wise learnable memory module is integrated into the mask predictor to capture and store class-wise geo-context at the instance level across the entire dataset. In addition, to address the absence of large-scale datasets for training pixel-level RS MLLMs, we construct the GeoPixInstruct dataset, comprising 65,463 images and 140,412 instances, with each instance annotated with text descriptions, bounding boxes, and masks. Furthermore, we develop a two-stage training strategy to balance the distinct requirements of text generation and masks prediction in multi-modal multi-task optimization. Extensive experiments verify the effectiveness and superiority of GeoPix in pixel-level segmentation tasks, while also maintaining competitive performance in image- and region-level benchmarks. The models, dataset, and code are publicly available at https://***/Norman-Ou/GeoPix. © 2025, CC BY.

关键词： Image segmentation

Isoform Function Prediction Based on Heterogeneous Graph Attention Networks

学校读者我要写书评

暂无评论

Isoform Function Prediction Based on Heterogeneous Graph Att...

2023 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2023

作者： Guo, Kuo Li, Yifan Chen, Hao Shen, Hong-Bin Yang, Yang Shanghai Jiao Tong University Key Lab. of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Department of Computer Science and Engineering Shanghai200240 China Shanghai Jiao Tong University Key Laboratory of System Control and Information Processing Ministry of Education of China Institute of Image Processing and Pattern Recognition Shanghai200240 China Carnegie Mellon University School of Computer Science Computational Biology Department PittsburghPA15213 United States

ISBN: (纸本)9798350337488

Isoforms refer to different mRNA molecules transcribed from the same gene, which can be translated into proteins with varying structures and functions. Predicting the functions of isoforms is an essential topic in bioinformatics as it can provide valuable insights into the intricate mechanisms of gene regulation and biological processes. Conventionally, gene function labels are standardized in Gene Ontology (GO) terms. However, traditional methods for predicting isoform function are largely limited by the absence of isoform-specific labels, sparse annotations, and the vast number of GO terms. To address these issues, we propose HANIso, a deep learning-based method for isoform function prediction. HANIso leverages a pretrained protein language model to extract features from protein sequences. It also integrates heterogeneous information, such as isoform sequence features, GO annotations, and isoform interaction data, using a Heterogeneous Graph Attention Network (HAN). This allows the model to learn the importance of different sources of information and their semantic relationships through the attention mechanism. Our method can predict function labels at both the gene level and isoform level. We conduct experiments on two species datasets, and the results demonstrate that our method outperforms existing methods on both AUROC and AUPRC. HANIso has the potential to overcome the limitations of traditional methods and provide a more accurate and comprehensive understanding of isoform function. © 2023 IEEE.

关键词： alternative splicing gene ontology heterogeneous graph attention network isoform function prediction protein language model

SliceProp: A Slice-Wise Bidirectional Propagation Model for Interactive 3D Medical Image Segmentation 1

学校读者我要写书评

暂无评论

SliceProp: A Slice-Wise Bidirectional Propagation Model for ...

1st IEEE International Conference on Medical Artificial Intelligence, MedAI 2023

作者： Xu, Xin Lu, Wenjing Lei, Jiahao Qiu, Peng Shen, Hong-Bin Yang, Yang Shanghai Jiao Tong University Key Lab. of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Department of Computer Science and Engineering Shanghai200240 China Shanghai Ninth People's Hospital Shanghai Jiao Tong University School of Medicine Department of Vascular Surgery China Shanghai Jiao Tong University Institute of Image Processing and Pattern Recognition Key Laboratory of System Control and Information Processing Ministry of Education of China Shanghai200240 China

ISBN: (纸本)9798350358780

Interactive medical image segmentation methods have become increasingly popular in recent years. These methods combine manual labeling and automatic segmentation, reducing the workload of annotation while maintaining high accuracy. However, most current interactive segmentation frameworks are limited to 2D image data, and are not suitable for 3D image data due to the large size and high complexity of 3D data, as well as the challenges posed by information asymmetry and sparse annotation. In this paper, we propose SliceProp, an interactive segmentation framework that implements slice-wise label Bidirectional Propagation (LBP) for 3D medical image segmentation. SliceProp extends the interactive 2D image segmentation algorithm to 3D image segmentation, and can handle 3D data with large size and high complexity. Moreover, equipped with a Backtracking Feedback Check (BFC) module, SliceProp effectively addresses the issues of information asymmetry and spatial sparse annotation in 3D medical image segmentation. Additionally, we adopt an uncertainty-based criterion to pri-oritize the slices to be refined interactively, which enhances the efficiency of the interaction process by enabling the model to focus on the regions with the most unreliable predictions. SliceProp is evaluated on two datasets and achieves promising results compared to state-of-the-art methods. © 2023 IEEE.

关键词： Medical imaging

Tlrm: Task-level relation module for GNN-based few-shot learning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Guo, Yurong Ma, Zhanyu Li, Xiaoxu Dong, Yuan Pattern Recognition and Intelligent System Lab Beijing University of Posts and Telecommunications Beijing China Lanzhou University of Technology Lanzhou China

Recently, graph neural networks (GNNs) have shown powerful ability to handle few-shot classification problem, which aims at classifying unseen samples when trained with limited labeled samples per class. GNN-based few-shot learning architectures mostly replace traditional metric with a learnable GNN. In the GNN, the nodes are set as the samples’ embedding, and the relationship between two connected nodes can be obtained by a network, the input of which is the difference of their embedding features. We consider this method of measuring relation of samples only models the sample-to-sample relation, while neglects the specificity of different tasks. That is, this method of measuring relation does not take the task-level information into account. To this end, we propose a new relation measure method, namely the task-level relation module (TLRM), to explicitly model the task-level relation of one sample to all the others. The proposed module captures the relation representations between nodes by considering the sample-to-task instead of sample-to-sample embedding features. We conducted extensive experiments on four benchmark datasets: mini-ImageNet, tiered-ImageNet, CUB-200-2011, and CIFAR-FS. Experimental results demonstrate that the proposed module is effective for GNN-based few-shot learning. © 2021, CC BY.

关键词： Graph neural networks

MRP-Net: A Light Multiple Region Perception Neural Network for Multi-label AU Detection

学校读者我要写书评

暂无评论

MRP-Net: A Light Multiple Region Perception Neural Network f...

International Conference on pattern recognition

作者： Yang Tang Shuang Chen Honggang Zhang Gang Wang Rui Yang Pattern Recognition and Intelligent System Lab. Beijing University of Posts and Telecommunications Beijing Anding Hospital Capital Medical University

Facial Action Units (AUs) are of great significance in communication. Automatic AU detection can improve the understanding of psychological condition and emotional status. Recently, a number of deep learning methods have been proposed to take charge with problems in automatic AU detection. Several challenges, like unbalanced labels and ignorance of local information, remain to be addressed. In this paper, we propose a fast and light neural network called MRP-Net, which is an end-to-end trainable method for facial AU detection to solve these problems. First, we design a Multiple Region Perception (MRP) module aimed at capturing different locations and sizes of features in the deeper level of the network without facial landmark points. Then, in order to balance the positive and negative samples in the large dataset, a batch balanced method adjusting the weight of every sample in one batch in our loss function is suggested. Experimental results on two popular AU datasets, BP4D and DISFA prove that MRP-Net outperforms state-of-the-art methods. Compared with the best method, not only does MRP-Net have an average F1 score improvement of 2.95% on BP4D and 5.43% on DISFA, and it also decreases the number of network parameters by 54.62% and the number of network FLOPs by 19.6%.

关键词： Performance evaluation Learning systems Deep learning Gold Neural networks Psychology Feature extraction

Meta Balanced Network for Fair Face recognition

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wang, Mei Zhang, Yaobin Deng, Weihong The Pattern Recognition and Intelligent System Lab. School of Artificial Intelligence Beijing University of Posts and Telecommunications Beijing China Key Lab. of Trustworthy Distributed Computing and Service Ministry of Education Beijing University of Posts and Telecommunications Beijing China

Although deep face recognition has achieved impressive progress in recent years, controversy has arisen regarding discrimination based on skin tone, questioning their deployment into real-world scenarios. In this paper, we aim to systematically and scientifically study this bias from both data and algorithm aspects. First, using the dermatologist approved Fitzpatrick Skin Type classification system and Individual Typology Angle, we contribute a benchmark called Identity Shades (IDS) database, which effectively quantifies the degree of the bias with respect to skin tone in existing face recognition algorithms and commercial APIs. Further, we provide two skin-tone aware training datasets, called BUPT-Globalface dataset and BUPT-Balancedface dataset, to remove bias in training data. Finally, to mitigate the algorithmic bias, we propose a novel meta-learning algorithm, called Meta Balanced Network (MBN), which learns adaptive margins in large margin loss such that the model optimized by this loss can perform fairly across people with different skin tones. To determine the margins, our method optimizes a meta skewness loss on a clean and unbiased meta set and utilizes backward-on-backward automatic differentiation to perform a second order gradient descent step on the current margins. Extensive experiments show that MBN successfully mitigates bias and learns more balanced performance for people with different skin tones in face recognition. The proposed datasets are available at http://***/RFW/***. © 2022, CC BY.

关键词： Face recognition

Progressive co-attention network for fine-grained visual classification

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhang, Tian Chang, Dongliang Ma, Zhanyu Guo, Jun Pattern Recognition and Intelligent System Lab Beijing University of Posts and Telecommunications Beijing China Beijing Academy of Artificial Intelligence Beijing China

Fine-grained visual classification aims to recognize images belonging to multiple sub-categories within a same category. It is a challenging task due to the inherently subtle variations among highly-confused categories. Most existing methods only take an individual image as input, which may limit the ability of models to recognize contrastive clues from different images. In this paper, we propose an effective method called progressive co-attention network (PCA-Net) to tackle this problem. Specifically, we calculate the channel-wise similarity by encouraging interaction between the feature channels within same-category image pairs to capture the common discriminative features. Considering that complementary information is also crucial for recognition, we erase the prominent areas enhanced by the channel interaction to force the network to focus on other discriminative regions. The proposed model has achieved competitive results on three fine-grained visual classification benchmark datasets: CUB-200-2011, Stanford Cars, and FGVC Aircraft. © 2021, CC BY.

关键词： Classification (of information)