Optimal Client Sampling for Federated Learning
Authors: Wenlin Chen, Samuel Horváth, Peter Richtárik
TMLR 2022 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This section empirically evaluates our optimal client sampling method on standard federated datasets from LEAF (Caldas et al., 2018). We compare our method with 1) full participation where all available clients participate in each round; and 2) the baseline where participating clients are sampled uniformly from available clients in each round. |
| Researcher Affiliation | Academia | Wenlin Chen EMAIL Department of Engineering University of Cambridge Cambridge, CB2 1PZ, UK and Department of Empirical Inference Max Planck Institute for Intelligent Systems Tübingen, 72076, Germany Samuel Horváth EMAIL Department of Machine Learning Mohamed bin Zayed University of Artificial Intelligence Masdar City, Abu Dhabi, UAE Peter Richtárik EMAIL Computer, Electrical and Mathematical Science and Engineering Division King Abdullah University of Science and Technology Thuwal, 23955-6900, Saudi Arabia |
| Pseudocode | Yes | Algorithm 1 Optimal Client Sampling (OCS). Algorithm 2 Approximate Optimal Client Sampling (AOCS). Algorithm 3 Fed Avg with Optimal Client Sampling. |
| Open Source Code | Yes | Our code together with datasets can be found at https://github.com/Samuel Horvath/FL-optimal-client-sampling. |
| Open Datasets | Yes | This section empirically evaluates our optimal client sampling method on standard federated datasets from LEAF (Caldas et al., 2018). We first evaluate our method on the Federated EMNIST (FEMNIST) image dataset for image classification. We also evaluate our method on the Shakespeare text dataset for next character prediction. In Appendix G, we present an additional experiment on the Federated CIFAR100 dataset from LEAF. |
| Dataset Splits | No | For validation, we use the unchanged EMNIST validation set, which consists of 40, 832 images. The paper describes client sampling and local training batch sizes, but does not provide explicit train/test/validation splits for the overall datasets themselves. |
| Hardware Specification | No | We simulate the cross-device FL distributed setting and train our models using Tensor Flow Federated (TFF). No specific hardware (GPU/CPU models) is mentioned. |
| Software Dependencies | No | We simulate the cross-device FL distributed setting and train our models using Tensor Flow Federated (TFF). The version of TFF or any other software dependency is not specified. |
| Experiment Setup | Yes | In each communication round, n = 32 clients are sampled uniformly from the client pool, each of which then performs several SGD steps on its local training images for 1 epoch with batch size 20. For partial participation, the expected number of clients allowed to communicate their updates back to the master is set to m {3, 6}. We use vanilla SGD optimizers with constant step sizes for both clients and the master, with ηg = 1 and ηl tuned on a holdout set. For full participation and optimal sampling, it turns out that ηl = 2 3 is the optimal local step size for all three datasets. For uniform sampling, the optimal is ηl = 2 5 for Dataset 1 and ηl = 2 4 for Datasets 2 and 3. We set jmax = 4 and include the extra communication costs in our results. |