检索结果-内蒙古大学图书馆

geometry-sensitive semantic modeling in visual and visual-language domains for image captioning

ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE 2025年 147卷

作者： Zhu, Wencai Jiang, Zetao He, Yuting Guilin Univ Elect Technol Guangxi Key Lab Image & Graph Intelligent Proc Guilin 541004 Peoples R China

Transformer-based models with grid features as visual representations perform well in image captioning. However, the division and flattening operations increase the difficulty of capturing objects and their relationships via pure semantic modeling. Furthermore, the natural language generated by the current Transformer model still suffers from semantic overconcentration. In this paper, we aim to improve the attention modules in two ways to solve the above issues. We first propose a geometry-sensitive Self-Attention (GSSA) module, subdivide geometric signals in the visual domain into relative position and distance, and assist the semantic modeling process according to their unique characteristics. It compensates for the lack of objects and their relationships in the grid features. Then, we propose a geometry-sensitive Cross-Attention (GSCA) module, which perceives the source neighboring relationships between images and text in the visual-language domain from a geometric perspective and uses these relationships to adjust the semantic correspondences between the two dynamically. It spreads overly focused attention to surrounding grids to improve understanding of full image content during captioning. To prove our designs, we apply GSSA and GSCA to a standard Transformer to construct a novel geometry-sensitive Transformer Network (GSTNet), which conducts geometry-sensitive semantic modeling in visual and visual-language domains. Extensive experiments are conducted to verify the effectiveness of our proposal. The results show that our GSTNet achieves superior performance compared to many state-of-theart image captioning models on the Microsoft Common Objects in Context (MSCOCO) dataset. Besides, the generalization of GSTNet is also verified on the Flickr30k dataset.

关键词： Image captioning Grid features Transformer mechanism geometry-sensitive semantic modeling

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：