Personalized Federated Learning via Low-Rank Matrix Optimization
Authors: Ali Dadras, Sebastian U Stich, Alp Yurtsever
TMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We examine the convergence guarantees of the proposed method and present numerical experiments on training deep neural networks, demonstrating the empirical performance of the proposed method in scenarios where personalization is crucial. 5 Numerical Experiments In this section, we evaluate the performance of p FLMF. We compare the performance of p FLMF against several baselines, including Local training, Fed Avg (Mc Mahan et al., 2017), Fed Per (Arivazhagan et al., 2019), Fed Rep (Collins et al., 2021), APFL (Deng et al., 2020), CFL (Sattler et al., 2021), FLUTE (Liu et al., 2024b), Fed AS (Yang et al., 2024), Fed Alt (Pillutla et al., 2022), and p Fed FDA (Mc Laughlin & Su, 2024) by implementing p FLMF in the FL-Bench benchmark (Tan & Wang, 2024; Tan et al., 2023). The source code is available at https://github.com/Dadras Ali/p FLMF. We conducted experiments in five different setups: Setup (1) For the MNIST, CIFAR10, and CIFAR100 datasets, we split the data according to the Dirichlet distribution Dir(0.5) and Dir(1) across 100 clients. The labels distribution is shown in Figures 2c and 2d. Performance of the algorithms is shown in Table 1. |
| Researcher Affiliation | Academia | Ali Dadras EMAIL Umeå University, Sweden Sebastian U. Stich EMAIL CISPA Helmholtz Center, Germany Alp Yurtsever EMAIL Umeå University, Sweden |
| Pseudocode | Yes | Algorithm 1 Personalized Federated Learning via Matrix Factorization (p FLMF) |
| Open Source Code | Yes | The source code is available at https://github.com/Dadras Ali/p FLMF. |
| Open Datasets | Yes | Setup (1) For the MNIST, CIFAR10, and CIFAR100 datasets, we split the data according to the Dirichlet distribution Dir(0.5) and Dir(1) across 100 clients. ... Setup (4) We sampled a subset of clients, 30% of the total clients, from FEMNIST dataset without changing the underlying data distribution, then we removed clients with less than 10 data points. |
| Dataset Splits | Yes | Setup (1) For the MNIST, CIFAR10, and CIFAR100 datasets, we split the data according to the Dirichlet distribution Dir(0.5) and Dir(1) across 100 clients. ... All experiments have 75% train and 25% test data splits on each client s data. |
| Hardware Specification | No | The computations were enabled by the supercomputing resource Berzelius provided by National Supercomputer Centre at Linköping University and the Knut and Alice Wallenberg foundation. |
| Software Dependencies | No | We compare the performance of the proposed method against the baseline in scenarios where personalization is crucial, such as classification in clustered FL with label misalignment or non-homogeneous data distributions. by implementing p FLMF in the FL-Bench benchmark (Tan & Wang, 2024; Tan et al., 2023). |
| Experiment Setup | Yes | Model. We used a three-layer neural network, consisting of three linear layers, on the MNIST and FEMNIST datasets and a four-layer convolutional neural network, consisting of two convolutional layers followed by two linear layers, on the CIFAR10 and CIFAR100 datasets. For Fed Per and Fed Rep, we treated the last layer as the classifier, while in p FLMF, we factorized the entire model. Hyper-parameters. We consider partial participation with probability equal to 0.1. We set the batch size equal to 256 for all algorithms. We tried parameter r values of (6) is set to r {1, 5, 10, 15}. All experiments have 75% train and 25% test data splits on each client s data. We chose the best step size for each algorithm from the set {10 4, 10 3, 10 2, 10 1}. |