Differentially Private Federated Learning with Time-Adaptive Privacy Spending
Authors: Shahrzad Kianidehkordi, Nupur Kulkarni, Adam Dziedzic, Stark Draper, Franziska Boenisch
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our practical experiments on standard benchmark datasets support our theoretical results and show that, in practice, our algorithms improve the privacy-utility trade-offs compared to baseline schemes. ... Based on these theoretical insights, in Sec. 5 we experimentally benchmark our framework against the baselines and show that the global test accuracy achieved by our method surpasses that of the baselines for the FMNIST (Xiao et al., 2017), MNIST (Deng, 2012), Adult Income (Becker & Kohavi, 1996), and CIFAR10 datasets (Krizhevsky et al., 2009). |
| Researcher Affiliation | Academia | 1 Department of Electrical and Computer Engineering, University of Toronto 2 CISPA Helmholtz Center for Information Security |
| Pseudocode | Yes | Algorithm 1 The spend-as-you-go Method in Our Time-adaptive DP-FL Framework ... Algorithm 2 Iterative Training in Our Timeadaptive DP-FL Framework |
| Open Source Code | No | The paper mentions using the 'Opacus library (Yousefpour et al., 2021)' but does not provide a specific link or statement for the authors' own implementation described in this paper. |
| Open Datasets | Yes | Based on these theoretical insights, in Sec. 5 we experimentally benchmark our framework against the baselines and show that the global test accuracy achieved by our method surpasses that of the baselines for the FMNIST (Xiao et al., 2017), MNIST (Deng, 2012), Adult Income (Becker & Kohavi, 1996), and CIFAR10 datasets (Krizhevsky et al., 2009). |
| Dataset Splits | Yes | For the Fashion MNIST (FMNIST) and MNIST, we use a convolutional neural network (CNN) architecture from (Mc Mahan et al., 2017). For the Adult Income dataset, we use multi-layer perception from (Zhang et al., 2020). For the CIFAR10 dataset, we use a CNN architecture from He et al. (2016). We partition datasets across 100 clients in a non-IID manner using the Dirichlet distribution with a default parameter 0.1 (Zhang et al., 2023). ... Both FMNIST and MNIST datasets have a training set of 60,000 and a test set of 10,000 28x28 images, associated with 10 labels. The Adult Income dataset consists of 48,842 samples with 14 features and is split into a training set of 32,561 samples and a test set of 16,281 samples. The CIFAR10 dataset consists of 60,000 32x32 color images in 10 classes, with 6000 images per class. There are 50,000 training images and 10,000 test images. |
| Hardware Specification | Yes | We conduct our experiments in Python 3.11 using Pytorch leveraging the 4 L4 24 GB GPU. |
| Software Dependencies | No | We conduct our experiments in Python 3.11 using Pytorch leveraging the 4 L4 24 GB GPU. ... We use the Opacus library (Yousefpour et al., 2021). The paper mentions Python 3.11, Pytorch, and Opacus, but does not specify version numbers for Pytorch or Opacus. |
| Experiment Setup | Yes | Throughout our simulations, we use SGD optimizer and momentum equal to 0.9. We also use a Cosine Annealing learning rate scheduler from (Loshchilov & Hutter, 2016) for faster convergence. In Sec.5, we fix the spending-based sample rate (during spend mode) to q = 0.9 and the average clipping norm to c = 250. We consider the transition from saving round to spending round occurs in the middle of training. I.e., given the total number of rounds T = 25, we set Tgroup,1 = Tgroup,2, Tgroup,3 = 13. ... In Table 4 we summarize other hyperparameters, including learning rate (λ), number of clients (N), batch size (B), number of local epochs (L), and the saving-based sampling rates of clients from privacy groups 1, 2, and 3 (qgroup,1, qgroup,2, qgroup,3). |