Multi-task learning for object keypoints detection and classification

作     者：Jie Xu Lin Zhao Shanshan Zhang Chen Gong Jian Yang 

作者机构：PCA Lab Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education and Jiangsu Key Lab of Image and Video Understanding for Social Security School of Computer Science and Engineering Nanjing University of Science and Technology Nanjing 210094 China State Key Laboratory of Integrated Services Networks Xidian Univeristy Xi’an 710071 China 

出 版 物：《Pattern Recognition Letters》 

年 卷 期：2020年第130卷

页      面：182-188页

学科分类：08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

摘      要：Object keypoints detection and classification are both central research topics in computer vision . Due to their wide range potential applications in the real world, substantial efforts have been taken to advance their performance. However, these two related tasks are mainly treated separately in previous works. We argue that keypoints detection and classification can be complementary tasks and beneficial to each other. Knowing the category of a object is able to reduce the searching space of keypoints detection models and facilitate more precise localization . On the other hand, having the knowledge of object keypoints can make classification models pay more attention on areas that are more associated with the object, which will inevitably promote classification accuracy . Embracing this observation, we propose to model keypoints detection and classification in a multi-task learning framework. Specifically, a multi-task deep network is designed and trained to conduct both tasks, where we devise the model structure delicately to carry out sufficient training of both tasks. Extensive experiments are set up on the AIFASHION DATASET and Human3.6M DATASET to validate our proposal, we show that our algorithm outperforms separate models trained individually on each task.