Spurious Privacy Leakage in Neural Networks
Authors: Chenxiang Zhang, Jun Pang, Sjouke Mauw
TMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this work, we investigate the privacy impact of spurious correlation bias. We introduce spurious privacy leakage, a phenomenon in which spurious groups are significantly more vulnerable to privacy attacks than non-spurious groups. We observe that privacy disparity between groups increases in tasks with simpler objectives (e.g. fewer classes) due to spurious features. Counterintuitively, we demonstrate that spurious robust methods, designed to reduce spurious bias, fail to mitigate privacy disparity. Our analysis reveals that this occurs because robust methods can reduce reliance on spurious features for prediction, but do not prevent their memorization during training. Finally, we systematically compare the privacy of different model architectures trained with spurious data, demonstrating that, contrary to previous work, architectural choice can affect privacy evaluation. |
| Researcher Affiliation | Academia | Chenxiang Zhang EMAIL Department of Computer Science University of Luxembourg Jun Pang EMAIL Department of Computer Science University of Luxembourg Sjouke Mauw EMAIL Department of Computer Science University of Luxembourg |
| Pseudocode | No | The paper describes the methodologies and algorithms used (e.g., Li RA, DRO, DFR) in paragraph form and with mathematical equations, but does not present any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | We release the code at https://github.com/orientino/spurious-mia. |
| Open Datasets | Yes | We select the datasets that are used by the spurious correlation community (Yang et al., 2023): Waterbirds (Sagawa et al., 2019), Celeb A (Liu et al., 2015), FMo W (Koh et al., 2021), Multi NLI (Williams et al., 2017), and Civil Comments (Koh et al., 2021). |
| Dataset Splits | Yes | For each dataset, we train the shadow models using 50% of the sampled training data as in the Li RA algorithm. We ensure that the sampled subset maintains a similar group proportion as the original dataset by first sampling per group, and then combining all the sampled groups together. |
| Hardware Specification | Yes | All the experiments are run on our internal cluster with the GPU Tesla V100 16GB/32GB of memory. |
| Software Dependencies | No | For text datasets, BERT s bert-base-uncased model (Devlin et al., 2019) is used. ... We train target models using opacus (Yousefpour et al., 2021) ... All the models used are pretrained using the state-of-the-art recipe on the Image Net1K dataset from the timm library. The paper mentions specific software (BERT, opacus, timm) and models, but does not provide specific version numbers for these software libraries. |
| Experiment Setup | Yes | We perform hyperparameter optimization for each dataset using a grid search over learning rate (lr), weight decay (wd), and epochs. The grid search and its best hyperparameters are in Appendix B. ... For Waterbirds and Celeb A we search the learning rate between [1e-3, 1e-4] and weight decay [1e-1, 1e-2, 1e-3]. For FMo W the learning rate [1e-3, 3e-3, 1e-4, 3e-4], weight decay [1e-1, 1e-2, 1e-3], and epochs [20, 30, 40]. For Multi NLI the learning rate [1e-5, 3e-5], weight decay [1e-5, 1e-4]. For Civil Comments the learning rate [1e-5, 1e-6], weight decay [1e-3, 1e-4]. The best hyperparameters are reported at Table 3. |