检索结果-内蒙古大学图书馆

multi-label image recognition with attentive transformer-localizer module

multiMEDIA TOOLS AND APPLICATIONS 2022年第6期81卷 7917-7940页

作者： Nie, Lin Chen, Tianshui Wang, Zhouxia Kang, Wenxiong Lin, Liang South China Univ Technol Wushan Campus 381 Wushan Rd Guangzhou Guangdong Peoples R China Guangdong Univ Technol Higher Educ Mega Ctr 100 WaiHuan West Rd Guangzhou Peoples R China Univ Hong Kong Hong Kong Peoples R China Sun Yat Sen Univ Higher Educ Mega Ctr 137 WaiHuan East Rd Guangzhou Guangdong Peoples R China

Recently, remarkable progress on multi-label image classification has been achieved by locating semantic-agnostic image regions and extracting their features with deep convolutional neural networks. However, existing pipelines depend on the hypothesis region generation step, which typically brings about extra computational costs, e.g., generating hundreds of meaningless proposals and extracting their features. Moreover, the contextual dependencies among these localized regions are usually ignored or oversimplified during the learning and inference stages. To resolve these issues, we develop a novel attentive transformer-localizer (ATL) module that contains differential transformations (e.g., translation, scale), which can automatically discover the discriminative semantic-aware regions from input images in terms of multi-label recognition. This module can be flexibly incorporated with recurrent neural networks such as the long short-term memory (LSTM) network for memorizing and updating the contextual dependencies of the localized regions. We thus build a unified multi-label image recognition framework. Specifically, the ATL module is applied to progressively localize the attentive regions from the convolutional feature maps in a proposal-free manner, and the LSTM network sequentially predicts label scores for the localized regions and updates the parameters of the ATL module while capturing the global dependencies among these regions. To associate the localized regions with semantic labels over diverse locations and scales, we further design three constraints together with the ATL module. Extensive experiments and evaluations on two large-scale benchmarks (i.e., PASCAL VOC and Microsoft COCO) show that the proposed approach achieves superior performance over existing state-of-the-art methods in terms of both performance and efficiency.

关键词： multi-label image recognition Attention model Recurrent memory network image context learning Deep representation learning

来源：评论

学校读者我要写书评

暂无评论

Semantic-Aware Graph Matching Mechanism for multi-label image recognition

引用

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 2023年第11期33卷 6788-6803页

作者： Wu, Yanan Feng, Songhe Wang, Yang Beijing Jiaotong Univ Minist Educ Key Lab Big Data & Artificial Intelligence Transp Beijing 100044 Peoples R China Beijing Jiaotong Univ Sch Comp & Informat Technol Beijing 100044 Peoples R China Univ Manitoba Dept Comp Sci Winnipeg MB R3T 2N2 Canada

multi-label image recognition aims to predict a set of labels that present in an image. The key to deal with such problem is to mine the associations between image contents and labels, and further obtain the correct assignments between images and their labels. In this paper, we treat each image as a bag of instances, and formulate the task of multi-label image recognition as an instance-label matching selection problem. To model such problem, we propose an innovative Semantic-aware Graph Matching framework for multi-label image recognition (ML-SGM), in which Graph Matching mechanism is introduced owing to its good performance of excavating the instance and label relationship. The framework explicitly establishes category correlations and instance-label correspondences by modeling the relation among content-aware (instance) and semantic-aware (label) category representations, to facilitate multi-label image understanding and reduce the dependency of large amounts of training samples for each category. Specifically, we first construct an instance spatial graph and a label semantic graph respectively and then incorporate them into a constructed assignment graph by connecting each instance to all labels. Subsequently, the graph network block is adopted to aggregate and update all nodes and edges state on the assignment graph to form structured representations for each instance and label. Our network finally derives a prediction score for each instance-label correspondence and optimizes such correspondence with a weighted cross-entropy loss. Empirical results conducted on generic multi-label image recognition demonstrate the superiority of our proposed method. Moreover, the proposed method also shows advantages in multi-label recognition with partial labels and multi-label few-shot learning, as well as outperforms current state-of-the-art methods with a clear margin. Our code is available at https://***/yananwu0510/ML-SGM.

关键词： multi-label image recognition graph matching graph neural network multi-label learning with partial labels multi-label few-shot learning

来源：评论

学校读者我要写书评

暂无评论

A multi-label image recognition Algorithm Based on Spatial and Semantic Correlation Interaction 6th

A Multi-label Image Recognition Algorithm Based on Spatial a...

引用

6th Chinese Conference on Pattern recognition and Computer Vision (PRCV)

作者： Cheng, Jing Ji, Genlin Yang, Qinkai Hao, Junzhao Nanjing Normal Univ Sch Artificial Intelligence Sch Comp & Elect Informat Nanjing Peoples R China

ISBN: (纸本)9789819985487;9789819985494

multi-label image recognition (MLIR) approaches usually exploit label correlations to achieve good performance. Two types of label correlations principally studied, i.e., the spatial and semantic correlations. However, most of the existing algorithms for multi-label image recognition consider semantic correlations and spatial correlations respectively, and often require additional information support. Although some algorithms simultaneously capture the semantic and spatial correlations of labels, they ignore the intrinsic relationship between the two. Specifically, only considering spatial correlations will misidentify some difficult objects in the image. For example, different categories of objects with similar appearance and close distance are mistaken for the same category, and semantic correlations can constrain the error caused by spatial correlations. In this work, we propose a multi-label image recognition algorithm based on transformer, named Spatial and Semantic Correlation Interaction (SSCI). Transformer is used to model the internal relationship between spatial correlations and semantic correlations to improve the recognition ability of the model for difficult objects. Experiments on the public datasets MS-COCO, VOC2007 and VOC2012 show that the mAP values reach 84.1%, 95.0% and 95.4%, respectively. Compared with other MLIR algorithms, the proposed algorithm can significantly improve the recognition performance of multi-label images.

关键词： multi-label image recognition transformer semantic correlations spatial correlations

来源：评论

学校读者我要写书评

暂无评论

A multi-scale semantic attention representation for multi-label image recognition with graph networks

引用

NEUROCOMPUTING 2022年 491卷 14-23页

作者： Liang, Jun Xu, Feiteng Yu, Songsen South China Normal Univ Sch Software Foshan 528225 Peoples R China

multi-label image recognition is a basic and challenging task in computer vision and multimedia fields. Graph Convolutional Networks (GCNs) are often used to learn the multi-label semantic features and multi-label dependency. Although the label semantic features in GCNs can learn the global image visual representation well, they are rarely used on the local image regions. Therefore, we try to use GCNs to learn global and local features at the same time, and make a balance between them. In this paper, we give a multi-scale semantic attention model MS-SGA-GCN including three main modules (i.e., MS, SGA and GCN) for multi-label image recognition. The multi-Scale module (MS) utilizes feature maps of different sizes to obtain global features and have strong generalization capabilities. Semantic Guide Attention module (SGA) applies the label embeddings learned by GCNs to guide the generation of the cross modality class-specific attention maps, which can discover the locations of semantically related regions for each label. Experiments show that our model on two datasets MS-COCO and PASCAL VOC2007 separately achieves the classification accuracy by 83.4% and 94.2%, which has a competitive advantage over other mainstream models. (c) 2022 Elsevier B.V. All rights reserved.

关键词： multi-label image recognition Graph convolutional network multi-scale Semantic guide attention Global level Local level

来源：评论

学校读者我要写书评

暂无评论

STMG: Swin transformer for multi-label image recognition with graph convolution network

引用

NEURAL COMPUTING & APPLICATIONS 2022年第12期34卷 10051-10063页

作者： Wang, Yangtao Xie, Yanzhao Fan, Lisheng Hu, Guangxing Guangzhou Univ Sch Comp Sci & Cyber Engn Guangzhou Peoples R China Huazhong Univ Sci & Technol Wuhan Peoples R China

Vision Transformer (ViT) has achieved promising single-label image classification results compared to conventional neural network-based models. Nevertheless, few ViT related studies have explored the label dependencies in the multi-label image recognition field. To this end, we propose STMG that combines transformer and graph convolution network (GCN) to extract the image features and learn the label dependencies for multi-label image recognition. STMG consists of an image representation learning module and a label co-occurrence embedding module. Firstly, in the image representation learning module, to avoid computing the similarity between each two patches, we adopt Swin transformer instead of ViT to generate the image feature for each input image. Secondly, in the label co-occurrence embedding module, we design a two-layer GCN to adaptively capture the label dependencies to output the label co-occurrence embeddings. At last, STMG fuses the image feature and label co-occurrence embeddings to produce the image classification results with the commonly-used multi-label classification loss function and a L2-norm loss function. We conduct extensive experiments on two multi-label image datasets including MS-COCO and FLICKR25K. Experimental results demonstrate STMG can achieve better performance including the convergence efficiency and classification results compared to the state-of-the-art multi-label image recognition methods. Our code is open-sourced and publicly available on GitHub: https://***/lzHZWZ/STMG.

关键词： Swin transformer Graph convolution network multi-label image recognition

来源：评论

学校读者我要写书评

暂无评论

SST: Spatial and Semantic Transformers for multi-label image recognition

引用

IEEE TRANSACTIONS ON image PROCESSING 2022年 31卷 2570-2583页

作者： Chen, Zhao-Min Cui, Quan Zhao, Borui Song, Renjie Zhang, Xiaoqin Yoshie, Osamu Wenzhou Univ Coll Comp Sci & Artificial Intelligence Wenzhou 325035 Peoples R China Waseda Univ Grad Sch Informat Prod & Syst Fukuoka 8080135 Japan Megvii Technol Megvii Res Nanjing Nanjing 210009 Peoples R China

multi-label image recognition has attracted considerable research attention and achieved great success in recent years. Capturing label correlations is an effective manner to advance the performance of multi-label image recognition. Two types of label correlations were principally studied, i.e., the spatial and semantic correlations. However, in the literature, previous methods considered only either of them. In this work, inspired by the great success of Transformer, we propose a plug-and-play module, named the Spatial and Semantic Transformers (SST), to simultaneously capture spatial and semantic correlations in multi-label images. Our proposal is mainly comprised of two independent transformers, aiming to capture the spatial and semantic correlations respectively. Specifically, our Spatial Transformer is designed to model the correlations between features from different spatial positions, while the Semantic Transformer is leveraged to capture the co-existence of labels without manually defined rules. Other than methodological contributions, we also prove that spatial and semantic correlations complement each other and deserve to be leveraged simultaneously in multi-label image recognition. Benefitting from the Transformer's ability to capture long-range correlations, our method remarkably outperforms state-of-the-art methods on four popular multi-label benchmark datasets. In addition, extensive ablation studies and visualizations are provided to validate the essential components of our method.

关键词： Correlation Semantics Transformers image recognition Task analysis Training Feature extraction multi-label image recognition transformer label correlation

来源：评论

学校读者我要写书评

暂无评论

label graph learning for multi-label image recognition with cross-modal fusion

引用

multiMEDIA TOOLS AND APPLICATIONS 2022年第18期81卷 25363-25381页

作者： Xie, Yanzhao Wang, Yangtao Liu, Yu Zhou, Ke Huazhong Univ Sci & Technol Wuhan Natl Lab Optoelect 1037 Luoyu Rd Wuhan Peoples R China Guangzhou Univ Sch Comp Sci & Cyber Engn 230 Wai Huan Xi Rd Guangzhou Peoples R China

It has become popular to learn the correlation between labels in most existing multi-label image recognition tasks. Existing approaches begin to construct a label graph to learn the label dependencies but they suffer from a low convergence efficiency when fusing image features and label embeddings, and also limit the performance improvement on multi-label images. To overcome this challenge, we propose a l abel g raph l earning m odel (termed as LGLM) for multi-label image recognition, which integrates a multi-modal fusion component to efficiently fuse cross-modal embeddings. First, LGLM uses convolution neural network to learn the feature for each image. Second, LGLM first constructs a label graph according to the word vector of each object and then adopts graph convolution network to learn the label correlations to generate label co-occurrence embeddings. Finally, the multi-modal fusion component efficiently fuses image features and label co-occurrence embeddings to generate an end-to-end image recognition model. We conduct extensive experiments on MS-COCO and FLICKR25K and the experimental results demonstrate the superiority of LGLM compared with the state-of-the-art image recognition methods. The code of LGLM has been released on GitHub: .

关键词： multi-label image recognition label graph Graph convolution network multi-modal fusion

来源：评论

学校读者我要写书评

暂无评论

SMART: Semantic-Aware Masked Attention Relational Transformer for multi-label image recognition

引用

IEEE SIGNAL PROCESSING LETTERS 2022年 29卷 2158-2162页

作者： Wu, Hongjun Xu, Cheng Liu, Hongzhe Beijing Union Univ Beijing Key Lab Informat Serv Engn Beijing 100101 Peoples R China

As objects usually co-exist in an image, learning the label co-occurrence is a compelling approach to improving the performance of multi-label image recognition. However, the dependencies among the non-exist categories in an image cannot be effectively evaluated. These redundant label dependencies may bring noise and further decrease the performance of classification. Therefore, we proposed SMART, a Semantic-aware Masked Attention Relational Transformer for multi-label image recognition for multi-label image recognition tasks. In addition to leveraging Transformer to model the inter-class dependencies, the proposed masked attention filters out the redundant dependencies among the non-exist categories. SMART is able to explicitly and accurately capture the label dependencies without extra word embeddings. Moreover, our method achieves new state-of-the-art results on two benchmarks for multi-label image recognition: MS-COCO 2014 and NUS-WIDE. In addition, extensive ablation studies and empirical analysis are provided to demonstrate the effectiveness of the essential components of our method under different factors.

关键词： Transformers Task analysis image recognition Semantics Correlation Convolution Visualization multi-label image recognition transformer label correlation masked attention

来源：评论

学校读者我要写书评

暂无评论

Dual-perspective semantic-aware representation blending for multi-label image recognition with partial labels

引用

EXPERT SYSTEMS WITH APPLICATIONS 2024年第PartA期249卷

作者： Pu, Tao Chen, Tianshui Wu, Hefeng Shi, Yukai Yang, Zhijing Lin, Liang Sun Yat sen Univ Sch Comp Sci & Engn Guangzhou 510006 Guangdong Peoples R China Guangdong Univ Technol Sch Informat Engn Guangzhou 510006 Guangdong Peoples R China

Recently, multi -label image recognition with partial labels (MLR -PL) has attracted increasing attention, in which only some labels are known while others are unknown for each image. However, current algorithms rely on pre -trained image similarity models or iteratively updating the image classification models to generate pseudo labels for unknown labels. Thus, they depend on a certain amount of annotations and inevitably suffer from obvious performance drops. To address this dilemma, we propose a dual -perspective semantic -aware representation blending (DSRB) framework that blends multi -granularity category -specific semantic representation across different images, from an instance and prototype perspective, respectively, to transfer information of known labels to complement unknown labels. Specifically, an instance -perspective representation blending (IPRB) module is designed to blend the representations of the known labels in an image with the representations of the corresponding unknown labels in another image to complement these unknown labels. Meanwhile, a prototype -perspective representation blending (PPRB) module is introduced to learn more stable representation prototypes for each category and blends the representation of unknown labels with the prototypes of corresponding labels in a location -sensitive manner to complement these unknown labels. Extensive experiments on various datasets show that the proposed DSRB consistently outperforms current state-of-the-art algorithms on all known label proportion settings.

关键词： multi-label image recognition Partial label learning Semi-supervised learning

来源：评论

学校读者我要写书评

暂无评论

Disentangling, Embedding and Ranking label Cues for multi-label image recognition

引用

IEEE TRANSACTIONS ON multiMEDIA 2021年 23卷 1827-1840页

作者： Chen, Zhao-Min Cui, Quan Wei, Xiu-Shen Jin, Xin Guo, Yanwen Nanjing Univ Natl Key Lab Novel Software Technol Nanjing Peoples R China Waseda Univ Grad Sch Informat Prod & Syst Tokyo 1698050 Japan Nanjing Univ Sci & Technol Sch Comp Sci & Engn Key Lab Intelligent Percept & Syst High Dimens In PCA LabMinist Educ Nanjing 210094 Peoples R China Nanjing Univ Sci & Technol Sch Comp Sci & Engn Jiangsu Key Lab Image & Video Understanding Socia Nanjing 210094 Peoples R China Megvii Technol Megvii Res Nanjing Nanjing 210000 Peoples R China Nanjing Lanzhong Intelligent Technol Co Ltd Nanjing 210018 Peoples R China

multi-label image recognition is a fundamental but challenging computer vision and multimedia task. Great progress has been achieved by exploiting label correlations among these multiple labels associated with a single image, which is the most crucial issue for multi-label image recognition. In this paper, to explicitly model label correlations, we propose a unified deep learning framework to Disentangle, Embed and Rank (DER) the corresponding label cues. Specifically, we first obtain class-aware disentangled maps (CADMs) by reforming deep activations in accordance with the class-specific recognition weights. Then, after transforming CADMs into the corresponding label vectors, we propose an embedding operation from a metric learning perspective to pull the relevant label vectors together and push irrelevant label vectors away. Furthermore, a ranking operation is employed, which aims to accurately and robustly measure the similarity/dissimilarity of these label vectors. Our model can be trained in an end-to-end manner with only image-level supervision, during which the proposed embedding and ranking operations can contribute to the CADMs learning through back-propagation. In addition, the obtained CADMs are aggregated and further used as an essential feature stream for the final multi-label classification. We conduct extensive experiments on three commonly used multi-label benchmark datasets. Quantitative results show that our model can significantly and consistently outperform previous competitive methods. Moreover, qualitative analysis of our DER proposal also reveals the effectiveness of our proposed model.

关键词： Correlation image recognition Streaming media Recurrent neural networks Task analysis Computational modeling Measurement multi-label image recognition deep learning label correlation CNNs disentangling embedding ranking

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：