Personalized Federated Learning with Communication Compression
Authors: El houcine Bergou, Konstantin Pavlovich Burlachenko, Aritra Dutta, Peter Richtárik
TMLR 2023 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To empirically validate the efficiency of our algorithm, we perform diverse numerical experiments on both convex and non-convex problems and use various compression techniques.1. We conducted diverse numerical experiments with L2GD algorithm that includes: (i) Analysis of algorithm meta-parameters (with and without compression) for logistic regression in strongly convex setting; see 7.1; (ii) analysis of compressed L2GD algorithm on image classification with DNNs; see 7.2. |
| Researcher Affiliation | Academia | El Houcine Bergou EMAIL College of Computing Mohammed VI Polytechnic University Ben Guerir, Morocco; Konstantin Burlachenko EMAIL King Abdullah University of Science and Technology, KSA; Aritra Dutta EMAIL Artificial Intelligence Initiative University of Central Florida Orlando, Florida-32816; Peter Richtárik EMAIL King Abdullah University of Science and Technology, KSA |
| Pseudocode | Yes | Algorithm 1 Compressed L2GD |
| Open Source Code | Yes | 1Our repository is available online: https://github.com/burlachenkok/compressed-fl-l2gd-code. |
| Open Datasets | Yes | We used L2GD algorithm with and without compression for solving ℓ2 regularized logistic regression on LIBSVM a1a and a2a datasets Chang & Lin (2011). We consider CIFAR-10 dataset (Krizhevsky & Hinton, 2009) for image classification. |
| Dataset Splits | Yes | The training and the test set are of size, 5 × 104 and 104, respectively. The training set is partitioned heterogeneously across 10 clients. The proportion of samples of each class stored at each local node is drawn by using the Dirichlet distribution (α = 0.5). |
| Hardware Specification | Yes | We performed experiments on server-grade machines running Ubuntu 18.04 and Linux Kernel v5.4.0, equipped with 8-cores 3.3 GHz Intel Xeon and a single NVIDIA Ge Force RTX 2080 Ti.Tesla-V100-SXM2 GPU with 32GB of GPU memory. |
| Software Dependencies | No | The computation backend for Logistics Regression experiments was Num Py library with leveraging MPI4PY for inter-node communication. For DNNs we used recent version of Fed ML He et al. (2020) benchmark4. The text does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | We set L2 = 1, and varied meta-parameters p and λ. For the Fed Avg algorithm, each client performs one epoch over the local data. We empirically tried 1, 2, 3, and 4 epochs over the local data as local steps, but one epoch is empirically the best choice. The step-sizes for Fed Avg and Fed Opt tuned via selecting step sizes from the following set {0.01, 0.1, 0.2, 0.5, 1.0, 2.0, 4.0}. We consider the step size for both algorithms to be 0.1. ... The batch size is set to 256. |