Visual Privacy Auditing with Diffusion Models
Authors: Kristian Schwethelm, Johannes Kaiser, Moritz Knolle, Sarah Lockfisch, Daniel Rueckert, Alexander Ziller
TMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this work, we empirically investigate this discrepancy by introducing a reconstruction attack based on diffusion models (DMs) that only assumes adversary access to real-world image priors and specifically targets the DP defense. We find that (1) real-world data priors significantly influence reconstruction success, (2) current reconstruction bounds do not model the risk posed by data priors well, and (3) DMs can serve as heuristic auditing tools for visualizing privacy leakage. Our experimentation includes three datasets: CIFAR-10 (Krizhevsky & Hinton, 2009), Celeb A-HQ (Karras et al., 2018), and Image Net-1K (Deng et al., 2009), with the latter two resized to 256 256. For evaluation, we randomly select a subset of 5,000 test images from each dataset and quantitatively measure the reconstruction success with mean squared error (MSE), VGG-based learned perceptual image patch similarity (LPIPS) (Simonyan & Zisserman, 2015; Zhang et al., 2018), and structural similarity index measure (SSIM) (Wang et al., 2004). |
| Researcher Affiliation | Academia | Kristian Schwethelm EMAIL Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital Johannes Kaiser EMAIL Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital Moritz Knolle EMAIL Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital Sarah Lockfisch EMAIL Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital Daniel Rückert EMAIL Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital Department of Computing, Imperial College London, UK Munich Center for Machine Learning (MCML), Munich, Germany Alexander Ziller EMAIL Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital |
| Pseudocode | Yes | Algorithm 1 summarizes our method. Algorithm 1: Private Image Reconstruction with DMs Require: xpriv = 1/λx + ξ, with ξ N(0, C2σ2I), noise schedule αt, model θ 1 αt 1 Variance schedule 2: x priv = λxpriv Rescaling 3: tstart = arg min t (σt Cσλ) σt > Cσλ Markov chain matching 4: xtstart = 1 p 1+σ2 tstart xpriv Reparameterization 5: for t = tstart, . . . , 1 do Step-wise denoising 6: xt 1 = αt 1 xt 1 αt ϵ(t) θ (xt) αt + 1 αt 1 ϵ(t) θ (xt), |
| Open Source Code | No | The paper references third-party libraries (Diffusers library, PyTorch, scikit-image) that are open-source and provides their citations, but there is no explicit statement about releasing the authors' own code for the methodology described in this specific paper, nor a direct link to their repository. |
| Open Datasets | Yes | Our experimentation includes three datasets: CIFAR-10 (Krizhevsky & Hinton, 2009), Celeb A-HQ (Karras et al., 2018), and Image Net-1K (Deng et al., 2009), with the latter two resized to 256 256. (1) The DM is trained on CIFAR-10 and is used to reconstruct test images from CIFAR-100 (Krizhevsky & Hinton, 2009). (3) the Image Net DM reconstructs grayscale chest X-ray images from the Che Xpert dataset (Irvin et al., 2019). |
| Dataset Splits | Yes | For evaluation, we randomly select a subset of 5,000 test images from each dataset and quantitatively measure the reconstruction success with mean squared error (MSE), VGG-based learned perceptual image patch similarity (LPIPS) (Simonyan & Zisserman, 2015; Zhang et al., 2018), and structural similarity index measure (SSIM) (Wang et al., 2004). We note that the employed DM s are not trained on test images. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory amounts) used for running the experiments. |
| Software Dependencies | No | Frameworks. We use the Diffusers library (von Platen et al., 2022) (based on Py Torch (Paszke et al., 2019)) to leverage state-of-the-art pre-trained DMs for implementing our reconstruction attack. In this experiment, we assess the impact of unknown noise variance on our reconstruction performance. For this, we approximate the noise variance using the waveletbased implementation in scikit-image (van der Walt et al., 2014) (restoration.estimate_sigma), which is described in Section 4.2 of (Donoho & Johnstone, 1994). The paper mentions software libraries (Diffusers, PyTorch, scikit-image) but does not provide specific version numbers for these dependencies. |
| Experiment Setup | Yes | We report results with respect to µ = C/σ, where C denotes the clipping parameter and σ the noise multiplier of DP-SGD. It can be interpreted as a signal-to-noise ratio (SNR), where C bounds the signal amplitude and σ represents the noise. Our experimental setup employs mini-batches of size 64 and a Res Net-9 architecture (Klause et al., 2022) augmented with an imprint layer (Fowl et al., 2022). The imprint layer s parameters are carefully tuned to separate individual activations within the accumulated gradient using a binning technique (128 bins). |