![]() We prove that SCAFFOLD requires significantly fewer communication rounds and is not affected by data heterogeneity or client sampling. ![]() As a solution, we propose a new algorithm (SCAFFOLD) which uses control variates (variance reduction) to correct for the ‘client drift’. We obtain a tight characterization of the convergence of FedAvg and prove that heterogeneity (non-iid-ness) in the client’s data results in a ‘drift’ in the local updates resulting in poor performance. The standard optimization algorithm used in this setting is Federated Averaging (FedAvg) due to its low communication cost. %X Federated learning is a key scenario in modern large-scale machine learning where the data remains distributed over a large number of clients and the task is to learn a centralized model without transmitting the client data. %C Proceedings of Machine Learning Research %B Proceedings of the 37th International Conference on Machine Learning %T SCAFFOLD: Stochastic Controlled Averaging for Federated Learning The latter is the first result to quantify the usefulness of local-steps in distributed optimization.Ĭite this = Further, we show that (for quadratics) SCAFFOLD can take advantage of similarity in the client’s data yielding even faster convergence. ![]() Federated learning is a key scenario in modern large-scale machine learning where the data remains distributed over a large number of clients and the task is to learn a centralized model without transmitting the client data.
0 Comments
Leave a Reply. |