FastFace: Fast-converging Scheduler for Large-scale Face Recognition Training with One GPU

作     者：Gong, Xueyuan Liu, Zhiquan Si, Yain-Whar Yuan, Xiaochen Wang, Ke Liu, Xiaoxiang Lin, Cong Zhang, Xinyuan 

作者机构：School of Intelligent Systems Science and Engineering Jinan University Zhuhai China College of Cyber Security Jinan University Guangzhou China Faculty of Science and Technology University of Macau China Faculty of Applied Sciences Macau Polytechnic University China College of Information Science and Technology Jinan University Guangzhou China 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2024年

核心收录：

主　　题：Face recognition 

摘      要：Computing power has evolved into a foundational and indispensable resource in the area of deep learning, particularly in tasks such as Face Recognition (FR) model training on large-scale datasets, where multiple GPUs are often a necessity. Recognizing this challenge, some FR methods have started exploring ways to compress the fully-connected layer in FR models. Unlike other approaches, our observations reveal that without prompt scheduling of the learning rate (LR) during FR model training, the loss curve tends to exhibit numerous stationary subsequences. To address this issue, we introduce a novel LR scheduler leveraging Exponential Moving Average (EMA) and Haar Convolutional Kernel (HCK) to eliminate stationary subsequences, resulting in a significant reduction in converging time. However, the proposed scheduler incurs a considerable computational overhead due to its time complexity. To overcome this limitation, we propose FastFace, a fast-converging scheduler with negligible time complexity, i.e. O(1) per iteration, during training. In practice, FastFace is able to accelerate FR model training to a quarter of its original time without sacrificing more than 1% accuracy, making large-scale FR training feasible even with just one single GPU in terms of both time and space complexity. Extensive experiments validate the efficiency and effectiveness of FastFace. The code is publicly available at: https://***/amoonfana/FastFace © 2024, CC BY.