On Memorization in Diffusion Models
Authors: Xiangming Gu, Chao Du, Tianyu Pang, Chongxuan Li, Min Lin, Ye Wang
TMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In response to these inquiries and concerns, this paper presents a comprehensive empirical study on memorization behavior in widely adopted diffusion models, including EDM (Karras et al., 2022) and Stable Diffusion (Rombach et al., 2022). We start with an analysis of EDM, noting that memorization tends to occur when trained on smaller-sized datasets, while remaining undetectable on larger datasets. This motivates our definition of effective model memorization (EMM), a metric quantifying the maximum number of training data points (sampled from distribution P) at which a diffusion model M demonstrates the similar memorization behavior as the theoretical optimum after the training procedure T . We then quantify the impact of critical factors on memorization in terms of EMM on the CIFAR-10, FFHQ (Karras et al., 2019), and Imagenette (Deng et al., 2009; Somepalli et al., 2023b) datasets, considering the three facets of P, M, and T . Besides comprehensive empirical results identifying the influential factors, we surprisingly find that conditioning training data on uninformative random labels can significantly trigger the memorization in diffusion models. |
| Researcher Affiliation | Collaboration | Xiangming Gu EMAIL National University of Singapore Chao Du EMAIL Sea AI Lab Tianyu Pang EMAIL Sea AI Lab Chongxuan Li EMAIL Gaoling School of Artificial Intelligence Renmin University of China Min Lin EMAIL Sea AI Lab Ye Wang EMAIL National University of Singapore |
| Pseudocode | No | The paper describes methods using mathematical equations and textual descriptions but does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our study holds practical significance for diffusion model users and offers clues to theoretical research in deep generative models. Code is available at https://github.com/sail-sg/Diff Memorize. |
| Open Datasets | Yes | We quantify the impact of critical factors on memorization in terms of EMM on the CIFAR-10, FFHQ (Karras et al., 2019), and Imagenette (Deng et al., 2009; Somepalli et al., 2023b) datasets, considering the three facets of P, M, and T . |
| Dataset Splits | No | The paper describes generating training datasets of different sizes ({20k, 10k, 5k, 2k}) by sampling subsets from the 50k CIFAR-10 training images and evaluating on 10k generated images. However, it does not provide explicit training/validation/test splits (e.g., 80/10/10%) for a single dataset used to reproduce the experiments in a conventional manner. |
| Hardware Specification | Yes | All experiments were run on 8 NVIDIA A100 GPUs, each with 80GB of memory. |
| Software Dependencies | No | The paper mentions using the Adam optimizer (Kingma & Ba, 2014) and model architectures like DDPM++ and U-Net, but it does not specify version numbers for any software libraries, programming languages, or specific frameworks like PyTorch or TensorFlow. |
| Experiment Setup | Yes | Our diffusion models are trained using Adam optimizer (Kingma & Ba, 2014) with a learning rate of 2 10 4 and a batch size of 512 for CIFAR-10 while 256 for FFHQ. The training duration is 40k epochs, while it is 4k epochs in Karras et al. (2022). ... Afterwards, the learning rate is fixed to 2 10 4 and the EMA rate is fixed to 0.99929. |