FEDERATED LEARNING WITH DYNAMIC CLIENT ARRIVAL AND DEPARTURE: CONVERGENCE AND RAPID ADAPTATION VIA INITIAL MODEL CONSTRUCTION

作     者：Chang, Zhan-Lun Han, Dong-Jun Parasnis, Rohit Hosseinalipour, Seyyedali Brinton, Christopher G. 

作者机构：Elmore Family School of Electrical and Computer Engineering Purdue University West LafayetteIN47907 United States Department of Computer Science and Engineering Yonsei University Seoul Korea Republic of The Laboratory for Information and Decision Systems Massachusetts Institute of Technology CambridgeMA02139 United States Department of Electrical Engineering University at Buffalo-SUNY BuffaloNY14260 United States 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2024年

核心收录：

主　　题：Federated learning 

摘      要：While most existing federated learning (FL) approaches assume a fixed set of clients in the system, in practice, clients can dynamically leave or join the system depending on their needs or interest in the specific task. This dynamic FL setting introduces several key challenges: (1) the objective function dynamically changes depending on the current set of clients, unlike traditional FL approaches that maintain a static optimization goal;(2) the current global model may not serve as the best initial point for the next FL rounds and could potentially lead to slow adaptation, given the possibility of clients leaving or joining the system. In this paper, we consider a dynamic optimization objective in FL that seeks the optimal model tailored to the currently active set of clients. Building on our probabilistic framework that provides direct insights into how the arrival and departure of different types of clients influence the shifts in optimal points, we establish an upper bound on the optimality gap, accounting for factors such as stochastic gradient noise, local training iterations, non-IIDness of data distribution, and deviations between optimal points caused by dynamic client pattern. We also propose an adaptive initial model construction strategy that employs weighted averaging guided by gradient similarity, prioritizing models trained on clients whose data characteristics align closely with the current one, thereby enhancing adaptability to the current clients. The proposed approach is validated on various datasets and FL algorithms, demonstrating robust performance across diverse client arrival and departure patterns, underscoring its effectiveness in dynamic FL environments. © 2024, CC BY.