A sketch recognition method based on bi-modal model using cooperative learning paradigm

作     者：Zhang, Shihui Wang, Lei Cui, Zhiguo Wang, Shi 

作者机构：School of Information Science and Engineering Yanshan University Hebei Street West Section Hebei Province Qinhuangdao066000 China Key Laboratory for Computer Virtual Technology and System Integration of Hebei Province Yanshan University Hebei Street West Section Hebei Province Qinhuangdao066000 China School of Information Technology Hebei University of Business and Economics Xuefu Road Hebei Province Shijiazhuang050061 China School of Mathematics and Information Science and Technology Hebei Normal University of Science and Technology Hebei Street West Section Hebei Province Qinhuangdao066000 China 

出 版 物：《Neural Computing and Applications》 (Neural Comput. Appl.)

年 卷 期：2024年第36卷第23期

页      面：14275-14290页

核心收录：

学科分类：0810[工学-信息与通信工程] 1205[管理学-图书情报与档案管理] 08[工学] 0701[理学-数学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：We are very grateful to the editors and reviewers for their time and efforts while reviewing this manuscript. Besides  we also appreciate the support of the Central Government Guided Local Funds for Science and Technology Development (No. 216Z0301G)  the National Natural Science Foundation of China (No. 61379065)  the Natural Science Foundation of Hebei Province in China (No. F2023203012)  and Innovation Capability Improvement Plan Project of Hebei Province (22567626 H) 

主　　题：Convolution 

摘      要：Static image is an important form of displaying a sketch, representing the appearance information of the sketch. And a stroke sequence composed of several points can also express the shape and contour information of the sketch. Therefore, it is very reasonable to treat a sketch as point-modal data and image-modal data simultaneously. In this paper, a method based on bi-modal model using cooperative learning paradigm is proposed for the sketch recognition task. Specifically, in the point-modal branch, a structural point convolution block is developed by properly dividing local regions to preserve the structural information. In the image-modal branch, the hierarchical residual structure is used to fully extract image-modal features. To reduce the negative impact of noisy samples on the recognition performance, a cooperative learning paradigm is designed based on different perceptual abilities of two modal branches on noisy samples, that is, when training the two branches, the noisy samples can be filtered out through information exchanges and mutual learning. Extensive experiments on the sketch datasets TU-Berlin and QuickDraw show that the proposed method outperforms most baseline methods and has many advantages such as no dependence on additional data and stroke information. © The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature 2024.