Exploring Inconsistent Knowledge Distillation for Object Detection with Data Augmentation

作     者：Liang, Jiawei Liang, Siyuan Liu, Aishan Ma, Ke Li, Jingzhi Cao, Xiaochun 

作者机构：School of Computer Science and Engineering Sun Yat-Sen University China State Key Laboratory of Information Security Institute of Information Engineering Chinese Academy of Sciences China NLSDE Beihang University China Institute of Dataspace Hefei China School of Electronic Electrical and Communication Engineering University of Chinese Academy of Sciences China School of Cyber Science and Technology Sun Yat-Sen University China 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2022年

核心收录：

主　　题：Object detection 

摘      要：Knowledge Distillation (KD) for object detection aims to train a compact detector by transferring knowledge from a teacher model. Since the teacher model perceives data in a way different from humans, existing KD methods only distill knowledge that is consistent with labels annotated by human expert while neglecting knowledge that is not consistent with human perception, which results in insufficient distillation and sub-optimal performance. In this paper, we propose inconsistent knowledge distillation (IKD), which aims to distill knowledge inherent in the teacher model’s counter-intuitive perceptions. We start by considering the teacher model’s counter-intuitive perceptions of frequency and non-robust features. Unlike previous works that exploit fine-grained features or introduce additional regularizations, we extract inconsistent knowledge by providing diverse input using data augmentation. Specifically, we propose a sample-specific data augmentation to transfer the teacher model’s ability in capturing distinct frequency components and suggest an adversarial feature augmentation to extract the teacher model’s perceptions of non-robust features in the data. Extensive experiments demonstrate the effectiveness of our method which outperforms state-of-the-art KD baselines on one-stage, two-stage and anchor-free object detectors (at most +1.0 mAP). Our codes will be made available at https://***/JWLiang007/***. Copyright © 2022, The Authors. All rights reserved.