PFLlib: A Beginner-Friendly and Comprehensive Personalized Federated Learning Library and Benchmark
Authors: Jianqing Zhang, Yang Liu, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, Jian Cao
JMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In PFLlib, we implemented 37 state-of-the-art FL algorithms (8 t FL algorithms and 29 p FL algorithms) and provided various evaluation environments with three statistically heterogeneous scenarios and 24 datasets. At present, PFLlib1 has gained more than 1600 stars and 300 forks on Git Hub. ... Benchmark. Due to limited space here, we only evaluate 20 algorithms in two label skew scenarios following the default settings10 of GPFL (Zhang et al., 2023b). Please refer to our oļ¬cial website11 for more documents, details, and results. In Table 2, we use the 4-layer CNN (Mc Mahan et al., 2017) for CV tasks on Fashion-MNIST (FMNIST) (Xiao et al., 2017), Cifar100, and Tiny-Image Net (Chrabaszcz et al., 2017) (TINY for short) datasets and use the fast Text (Joulin et al., 2017) for NLP tasks on AG News (Zhang et al., 2015) dataset. We also use Res Net-18 (He et al., 2016) on Tiny-Image Net and denote it TINY*. |
| Researcher Affiliation | Academia | Jianqing Zhang1 EMAIL Yang Liu2, EMAIL Yang Hua3 EMAIL Hao Wang4 EMAIL Tao Song1 EMAIL Zhengui Xue1 EMAIL Ruhui Ma1 EMAIL Jian Cao1, EMAIL 1 Shanghai Jiao Tong University, Shanghai, China 2 Institute for AI Industry Research, Tsinghua University, Beijing, China 3 Queen s University Belfast, Belfast, UK 4 Stevens Institute of Technology, New Jersey, USA Corresponding Authors |
| Pseudocode | No | The paper describes how to use the PFLlib library with command-line examples for scenario generation and algorithm evaluation, but it does not include a formal pseudocode block or algorithm description for any of the methods. |
| Open Source Code | Yes | At present, PFLlib1 has gained more than 1600 stars and 300 forks on Git Hub. Keywords: federated learning, personalization, privacy, benchmark, heterogeneity ... 1. https://www.pfllib.com/ and https://github.com/TsingZ0/PFLlib |
| Open Datasets | Yes | In PFLlib, we ... incorporated 24 datasets, covering Computer Vision (CV), Natural Language Processing (NLP), and Sensor Signal Processing (SSP) tasks. ... HAR (Anguita et al., 2012) and PAMAP2 (Reiss and Stricker, 2012) for SSP tasks), hospitals (Camelyon17 (Koh et al., 2021) for CV tasks), and camera traps (i Wild Cam (Koh et al., 2021) for CV tasks) ... FMNIST (Xiao et al., 2017), Cifar100, and Tiny-Image Net (Chrabaszcz et al., 2017) ... AG News (Zhang et al., 2015) dataset. |
| Dataset Splits | No | The paper mentions evaluating algorithms in "two label skew scenarios following the default settings10 of GPFL (Zhang et al., 2023b)" and notes that "some default settings and codes for scenario creation may change in PFLlib." While it refers to scenarios, it does not explicitly provide specific percentages, sample counts, or detailed methodologies for training/test/validation splits within the main text of this paper. |
| Hardware Specification | No | The paper describes the PFLlib library and benchmark platform and the algorithms and datasets implemented within it. However, it does not provide any specific details regarding the hardware (e.g., GPU models, CPU types) used to run the experiments or evaluations. |
| Software Dependencies | No | The paper introduces PFLlib as a library and mentions other related FL frameworks, but it does not list specific software dependencies (e.g., Python, PyTorch, TensorFlow) along with their version numbers required to run the library or reproduce the experiments. |
| Experiment Setup | Yes | Due to limited space here, we only evaluate 20 algorithms in two label skew scenarios following the default settings10 of GPFL (Zhang et al., 2023b). ... python main.py -data MNIST -m CNN -algo Fed ALA -gr 2000 -did 0 # in ./ system |