Adaptive gradient coding

作     者：Cao, Hankun Yan, Qifa Tang, Xiaohu Han, Guojun 

作者机构：Information Security and National Computing Grid Laboratory Southwest Jiaotong University Chengdu China The Electrical and Computer Engineering Department University of Illinois at Chicago ChicagoIL60607 United States The School of Information Engineering Guangdong University of Technology Guangzhou China 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2020年

核心收录：

主　　题：Machine learning 

摘      要：This paper focuses on mitigating the impact of stragglers in distributed learning system. Unlike the existing results designated for a fixed number of stragglers, we develop a new scheme called Adaptive Gradient Coding (AGC) with flexible communication cost for varying number of stragglers. Our scheme gives an optimal tradeoff between computation load, straggler tolerance and communication cost by allowing workers to send multiple signals sequentially to the master. In particular, it can minimize the communication cost according to the unknown real-time number of stragglers in practical environments. In addition, we present a Group AGC (G-AGC) by combining the group idea with AGC to resist more stragglers in some situations. The numerical and simulation results demonstrate that our adaptive schemes can achieve the smallest average running time. Copyright © 2020, The Authors. All rights reserved.