Decentralized Decoupled Training for Federated Long-Tailed Learning
Authors: Wenkai Yang, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun
TMLR 2024 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments show that our method consistently outperforms the existing state-of-the-art methods in various settings. Our code is available at https://github.com/keven980716/Federated_Learning_Experiments. We conduct extensive experiments on three popular long-tailed image classification tasks, and the results show that our method can significantly outperform all existing federated long-tailed learning methods in various settings. |
| Researcher Affiliation | Collaboration | Wenkai Yang EMAIL Gaoling School of Artificial Intelligence Renmin University of China; Deli Chen EMAIL Deep Seek AI; Hao Zhou EMAIL Pattern Recognition Center, We Chat AI Tencent Inc.; Xu Sun EMAIL National Key Laboratory for Multimedia Information Processing, School of Computer Science Peking University |
| Pseudocode | Yes | Algorithm 1 Local Training Process of Red Grape |
| Open Source Code | Yes | Our code is available at https://github.com/keven980716/Federated_Learning_Experiments. |
| Open Datasets | Yes | We conduct experiments on three popular image classification benchmarks: MNIST (Le Cun et al., 1998), CIFAR-10 and CIFAR-100 (Krizhevsky et al., 2009). We also conduct extra experiments on a large-scale realistic federated dataset FEMNIST (Cohen et al., 2017) |
| Dataset Splits | Yes | We follow existing studies (Cao et al., 2019; Shang et al., 2022b) to create the long-tailed versions of training sets of above three datasets (i.e., MNIST-LT, CIFAR-10/100-LT), and keep the test sets as balanced. We follow the existing studies (Reddi et al., 2020; Shang et al., 2022b) to adopt the Dirichlet distribution Dir(α) for the non-i.i.d. data partitioning, in which α controls the non-i.i.d. degree. We set α = 1.0 in our main experiments. |
| Hardware Specification | Yes | Our experiments are conducted on 8 * Ge Force RTX 2080 Ti. |
| Software Dependencies | No | Our code is implemented based on the open-sourced FL platform Fed ML (He et al., 2020). |
| Experiment Setup | Yes | We utilize SGDM as the local optimizer in all experiments. The final local learning rate is 0.01 for MNIST-LT, and 0.1 for CIFAR-10/100-LT. The tuned server learning rate is 1.0 for all settings. For all three datasets, the batch size for local training is 64, and the number of local training epochs is 5. As for our method, the re-balance factor λ is fixed as 0.1 in all experiments... The quantity threshold T for each class to create the local balanced dataset is set as 8 for MNIST-LT and CIFAR-10-LT, and 2 for CIFAR-100-LT |