Chinese Image Captioning via Fuzzy Attention-based DenseNet-BiLSTM

作     者：Lu, Huimin Yang, Rui Deng, Zhenrong Zhang, Yonglin Gao, Guangwei Lan, Rushi 

作者机构：Guilin Univ Elect Technol Guangxi Key Lab Image & Graph Intelligent Proc 1 Jinji Rd Guilin Peoples R China Kyushu Inst Technol Dept Mech & Control Engn Kitakyushu Fukuoka Japan Nanjing Univ Posts & Telecommun Inst Adv Technol 9 Wenyuan Rd Nanjing Peoples R China South China Univ Technol Sch Comp Sci & Engn Guangzhou Peoples R China 

出 版 物：《ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS》 (ACM Trans. Multimedia Comput. Commun. Appl.)

年 卷 期：2021年第17卷第1期

页      面：1–18页

核心收录：

学科分类：0809[工学-电子科学与技术（可授工学、理学学位）] 08[工学] 0835[工学-软件工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：National Key R&D Program of China [2018AAA0100300] National Natural Science Foundation of China [U1701267, 61772149, 61936002] Guangxi Science and Technology Project [ZY20198016, AB20238013, 2019GXNSFFA245014, AD18216004, AD18281079] Guangxi Key Laboratory of Image and Graphic Intelligent Processing [GIIP2003] GUET Excellent Graduate Thesis Program [18YJPYSS15] 

主　　题：Image captioning fuzzy attention DenseNet BiLSTM 

摘      要：Chinese image description generation tasks usually have some challenges, such as single-feature extraction, lack of global information, and lack of detailed description of the image content. To address these limitations, we propose a fuzzy attention-based DenseNet-BiLSTM Chinese image captioning method in this article. In the proposed method, we first improve the densely connected network to extract features of the image at different scales and to enhance the model s ability to capture the weak features. At the same time, a bidirectional LSTM is used as the decoder to enhance the use of context information. The introduction of an improved fuzzy attention mechanism effectively improves the problem of correspondence between image features and contextual information. We conduct experiments on the AI Challenger dataset to evaluate the performance of the model. The results show that compared with other models, our proposed model achieves higher scores in objective quantitative evaluation indicators, including BLEU@1, BLEU@4, METEOR, ROUGEl, and CIDEr. The generated description sentence can accurately express the image content.