Targeted Attack Improves Protection against Unauthorized Diffusion Customization
Authors: Boyang Zheng, Chumeng Liang, Xiaoyu Wu
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 4 EXPERIMENTS We compare ACE/ACE+/ACE to existing protections against unauthorized diffusion customization in two main diffusion customization methods, Lo RA+Dream Booth (Hu et al., 2021; Ruiz et al., 2023) and SDEdit (Meng et al., 2021). We also evaluate our protection on Dream Booth and Stable Diffusion 3 (Esser et al., 2024) (Appendix B.3). Our methods outperform baselines both in quantitative metrics and qualitative visualization (Section 4.2). We conduct user studies in an artist community, which validates the superiority of our methods (Section 4.3). Furthermore, we test the transferability (Section 4.4) and robustness to purification (Section 4.5) of ACE/ACE+. |
| Researcher Affiliation | Academia | Boyang Zheng1 Chumeng Liang2 Xiaoyu Wu1 1 Shanghai Jiao Tong University, 2 University of Southern California |
| Pseudocode | Yes | Algorithm 1 Attacking with Consistent Errors (ACE) 1: Input: Image x, diffusion model θ, learning rates α, γ, epoch numbers N, M, K, budget ζ, diffusion training objective in Equation 2, ACE/ACE+ objective function J in Equation 4 & Equation 5. 2: Output: Adversarial example x 3: Initialize x x. 4: for n from 1 to N do 5: for m from 1 to M do 6: θ θ γ θLLDM(x , θ) 7: end for 8: for k from 1 to K do 9: x x α x J 10: x clip(x , x ζ, x + ζ) 11: x clip(x , 0, 255) 12: end for 13: end for 14: return x |
| Open Source Code | No | The paper does not explicitly state that the authors' implementation code is open-source or provide a link to it. |
| Open Datasets | Yes | Datasets & Backbone Model The experiment is conducted on Celeb A-HQ (Karras et al., 2017) and Wikiart (Saleh & Elgammal, 2015). |
| Dataset Splits | No | The paper describes the selection of images for the datasets (100 images from Celeb A-HQ, 100 paintings from Wikiart, each grouped by 20), but does not provide explicit training, validation, or test dataset splits for their methodology. |
| Hardware Specification | Yes | All experiments except Photo Guard+ are done on one NVIDIA RTX 4090 GPU. The implementation of Photo Guard+ is done on one NVIDIA RTX A100 GPU. ... To better simulate the real-world scenario of artwork protection, this experiment is run on an NVIDIA RTX 3080Ti11, which is an off-the-shelf consumer-level GPU. |
| Software Dependencies | No | The paper mentions leveraging 'xFormers' for memory optimization but does not provide specific version numbers for xFormers or any other core software dependencies (e.g., Python, PyTorch, CUDA) used in their implementation. |
| Experiment Setup | Yes | Hyperparameters The adversarial budget ζ is set as 4/255. Note that this budget is smaller than those in existing literature, such as 8/255 Liang et al. (2023); Van Le et al. (2023) and 16/255 Salman et al. (2023), for these large budgets will add perceptible noise to the image that hurt the image quality. Hence, we use a smaller budget to simulate the real-world application scenario. The step length and the number of step in PGD (Madry et al., 2017) are 5 10 3 and 50. We adopt Lo RA for fine-tuning steps in Algorithm 1. Other hyper-parameters are omitted to Appendix A. ... Hyperparameters & Implementation Details For ACE+, the loss weight α is set to be 102 empirically. Lo RA is done for 5 iterations while each iteration finetunes 10 steps. The learning rate is 10 5. |