LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression
Authors: Laurent Condat, Artavazd Maranjyan, Peter Richtarik
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate the performance of our proposed method Lo Co DL and compare it with several other methods that also allow for CC and converge linearly to x . We also include Grad Skip (Maranjyan et al., 2022) and Scaffold (Mc Mahan et al., 2017) in our comparisons. We focus on a regularized logistic regression problem, which has the form (1) with ... We show the results with the a5a , diabetes , w1a datasets in Figures 1, 2, 3, respectively. |
| Researcher Affiliation | Academia | Laurent Condat, Artavazd Maranjyan & Peter Richtárik Computer Science Program, CEMSE Division King Abdullah University of Science and Technology (KAUST) Thuwal, 23955-6900, Kingdom of Saudi Arabia & SDAIA-KAUST Center of Excellence in Data Science and Artificial Intelligence (SDAIA-KAUST AI) EMAIL |
| Pseudocode | Yes | Algorithm 1 Lo Co DL |
| Open Source Code | No | The paper does not contain an explicit statement about releasing the source code for the Lo Co DL method, nor does it provide a link to a code repository. |
| Open Datasets | Yes | We considered several datasets from the Lib SVM library (Chang & Lin, 2011) (3-clause BSD license). We show the results with the a5a , diabetes , w1a datasets in Figures 1, 2, 3, respectively. ... Finally, we also run experiments on MNIST dataset (Le Cun et al., 1998) in Figure 6. |
| Dataset Splits | No | We prepared each dataset by first shuffling it, then distributing it equally among the n clients (since m in (11) is an integer, the remaining datapoints were discarded). This describes how data was distributed among clients but does not specify explicit training, validation, or test splits for evaluating the model. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU models, CPU types, memory) used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software components or libraries used in the experiments. |
| Experiment Setup | Yes | We focus on a regularized logistic regression problem, which has the form (1) with ... µ is the regularization parameter, set so that κ = 104. ... For all algorithms, we used the theoretical parameter values given in their available convergence results (Corollary 3.2 for Lo Co DL). We tried to tune the parameter values, such as k in rand-k and the (average) number of local steps per round, but this only gave minor improvements. For instance, ADIANA in Figure 1 was a bit faster with the best value of k = 20 than with k = 30. Increasing the learning rate γ led to inconsistent results, with sometimes divergence. |