Communication-Efficient Federated Bilevel Optimization with Global and Local Lower Level Problems
Authors: Junyi Li, Feihu Huang, Heng Huang
NeurIPS 2023 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we validate the proposed algorithms through two real-world tasks: Federated Datacleaning and Federated Hyper-representation Learning. Empirical results show superior performance of our algorithms. |
| Researcher Affiliation | Academia | Junyi Li Computer Science University of Maryland College Park, MD 20742 EMAIL Feihu Huang ECE University of Pittsburgh Pittsburgh, PA 15261 EMAIL Heng Huang Computer Science University of Maryland College Park, MD 20742 EMAIL |
| Pseudocode | Yes | Algorithm 1 Accelerated Federated Bilevel Optimization (Fed Bi OAcc) |
| Open Source Code | No | The paper mentions implementation details like Py Torch and Py Torch.Distributed but does not provide an explicit statement about releasing the source code for the described methodology or a link to a code repository. |
| Open Datasets | Yes | Dataset and Baselines. We create 10 clients and construct datasets based on MNIST [31]. For the training set, each client randomly samples 4500 images (no overlap among clients) from 10 classes and then randomly uniformly perturb the labels of ρ (0 ρ 1) percent samples. ... We consider the Omniglot [29] and Mini Image Net [44] data sets. |
| Dataset Splits | Yes | For the training set, each client randomly samples 4500 images (no overlap among clients) from 10 classes and then randomly uniformly perturb the labels of ρ (0 ρ 1) percent samples. For the validation set, each client randomly selects 50 clean images from a different class. |
| Hardware Specification | Yes | Our experiments were conducted on servers equipped with an AMD EPYC 7763 64-core CPU and 8 NVIDIA V100 GPUs. |
| Software Dependencies | No | The paper mentions 'Py Torch' and 'Py Torch.Distributed package' but does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | Input: Constants cω, cν, cu, γ, η, τ, r; learning rate schedule {αt}, t [T], initial state (x1, y1, u1); ... The local step I is set as 5 for Fed Bi O, Fed Bi OAcc and Fed Avg. ... We perform grid search to find the best hyper-parameters for each method and report the best results. Specific choices are included in Appendix B.1. |