This paper investigates efficient distributed training of a Federated Learning (FL) model over a wireless network of wireless devices. The communication iterations of the distributed training algorithm may be substant...
详细信息
This paper investigates efficient distributed training of a Federated Learning (FL) model over a wireless network of wireless devices. The communication iterations of the distributed training algorithm may be substantially deteriorated or even blocked by the effects of the devices' background traffic, packet losses, congestion, or latency. We abstract the communication-computation impacts as an 'iteration cost' and propose a cost-aware causal FL algorithm (FedCau) to tackle this problem. We propose an iteration-termination method that trade-offs the training performance and networking costs. We apply our approach when workers use the slotted-ALOHA, carrier-sense multiple access with collision avoidance (CSMA/CA), and orthogonal frequency-division multiple access (OFDMA) protocols. We show that, given a total cost budget, the training performance degrades as either the background communication traffic or the dimension of the training problem increases. Our results demonstrate the importance of proactively designing optimal cost-efficient stopping criteria to avoid unnecessary communication-computation costs to achieve a marginal FL training improvement. We validate our method by training and testing FL over the MNIST and CIFAR-10 dataset. Finally, we apply our approach to existing communication efficient FL methods from the literature, achieving further efficiency. We conclude that cost-efficient stopping criteria are essential for the success of practical FL over wireless networks.
A recursive algorithm for the determinant evaluation of general opposite-bordered tridiagonal matrices has been proposed by Jia et al. (J Comput Appl Math 290:423-432, 2015). Since the algorithm is a symbolic algorith...
详细信息
A recursive algorithm for the determinant evaluation of general opposite-bordered tridiagonal matrices has been proposed by Jia et al. (J Comput Appl Math 290:423-432, 2015). Since the algorithm is a symbolic algorithm, it never suffers from breakdown. However, it may be time-consuming when many symbolic names emerge during the symbolic computation. In this paper, without using symbolic computation, first we present a novel breakdown-free numerical algorithm for computing the determinant of ann-by-nopposite-bordered tridiagonal matrix, which does not require any extra memory storage for the implementation. Then, we present a cost-efficient algorithm for the determinants of opposite-bordered tridiagonal matrices based on the use of the combination of an elementary column operation and Sylvester's determinant identity. Furthermore, we provide some numerical results with simulations in Matlab implementation in order to demonstrate the accuracy and efficiency of the proposed algorithms, and their competitiveness with other existing algorithms. The corresponding results in this paper can be readily obtained for computing the determinants of singly-bordered tridiagonal matrices.
暂无评论