FedSMU: Communication-Efficient and Generalization-Enhanced Federated Learning through Symbolic Model Updates
Authors: Xinyi Lu, Hao Zhang, Chenglin Li, Weijia Lu, Zhifei Yang, Wenrui Dai, Xiaodong Zhang, Xiaofeng Ma, Can Zhang, Junni Zou, Hongkai Xiong
ICML 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through extensive experimental evaluations on several benchmark datasets, we demonstrate that our Fed SMU algorithm not only reduces the communication overhead, but also achieves a better generalization performance than the other compression-based and optimization-based baselines. ... We conduct a series of experiments to demonstrate the superiority of Fed SMU. |
| Researcher Affiliation | Collaboration | 1Shanghai Jiao Tong University, Shanghai, China. 2United Automotive Electronic Systems, Shanghai, China. |
| Pseudocode | Yes | Algorithm 1 Federated learning through Symbolic Model Updates (Fed SMU) algorithm. |
| Open Source Code | Yes | The implementable code of our proposed Fed SMU algorithm is available at https://github.com/lxy66888/fedsmu.git. |
| Open Datasets | Yes | We evaluate our Fed SMU and the other baseline algorithms on three real-world visual and language datasets: CIFAR-10, CIFAR-100 (Krizhevsky et al., 2009) and neural machine translation on Shakespeare, with the same train/test splits as in (Acar et al., 2021). ... The Tiny-Image Net dataset, a reduced version of the ILSVRC (Image Net Large Scale Visual Recognition Challenge) (Russakovsky et al., 2015). |
| Dataset Splits | Yes | We evaluate our Fed SMU and the other baseline algorithms on three real-world visual and language datasets: CIFAR-10, CIFAR-100 (Krizhevsky et al., 2009) and neural machine translation on Shakespeare, with the same train/test splits as in (Acar et al., 2021). ... Both of them comprise 50,000 images for training and 10,000 images for testing. ... Each category includes 500 images for training and 50 for testing. |
| Hardware Specification | Yes | All approaches are implemented in Py Torch 1.4.0 and CUDA 9.2, with GEFORCE GTX 1080 Ti throughout our experiments. |
| Software Dependencies | Yes | All approaches are implemented in Py Torch 1.4.0 and CUDA 9.2, with GEFORCE GTX 1080 Ti throughout our experiments. |
| Experiment Setup | Yes | The learning rates and hyperparameters for all approaches are individually tuned via a grid search. For additional details on hyperparameter settings, please refer to Appendix A. ... For local update in all methods, we tune the local learning rate over {1, 0.1, 0.01, 0.001} and set up 5 epochs of local updates with the minibatch B = 50. ... For our proposed method Fed SMU, we tune the parameter β1 and β2 over {0.9, 0.99, 0.999}, respectively, and set them both to 0.9 for CIFAR-10, CIFAR-100 and Tiny-Image Net, and 0.95 for Shakespeare. We tune the parameter γ1 and γ2 over {1, 0.1, 0.02, 0.018, 0.015, 0.013, 0.01, 0.005, 0.001}, respectively, since they are so sensitive, and set them to 0.015, 0.01 for CIFAR-10, 0.018, 0.01 for CIFAR-100, 0.01, 0.01 for Tiny-Image Net, and 0.03, 0.01 for Shakespeare. |