DuMo: Dual Encoder Modulation Network for Precise Concept Erasure
Authors: Feng Han, Kai Chen, Chao Gong, Zhipeng Wei, Jingjing Chen, Yu-Gang Jiang
AAAI 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our method achieves state-of-the-art performance on Explicit Content Erasure (detecting only 34 nude parts), Cartoon Concept Removal (with an average LPIPSda of 0.428, 0.113 higher than SOTA at 0.315), and Artistic Style Erasure (with an average LPIPSda of 0.387, 0.088 higher than SOTA at 0.299), clearly outperforming alternative methods. ... We conduct a variety of experiments on SD v1.4 to demonstrate the effectiveness of our method. Specifically, we compare ours with 10 baseline methods, including ESD (Gandikota et al. 2023), UCE (Gandikota et al. 2024), SLD-Med (Schramowski et al. 2023), SA (Heng and Soh 2024), CA (Kumari et al. 2023), SDD (Kim et al. 2023), RECE (Gong et al. 2024) MACE (Lu et al. 2024), and SPM (Lyu et al. 2024). Following SPM (Lyu et al. 2024), we perform three tasks: explicit content erasure (Sec. 4.1), cartoon concept removal (Sec. 4.2), and artistic style erasure (Sec. 4.3), to evaluate these methods. Besides, we conduct the ablation study (Sec. 4.4) to verify the effects of each component. |
| Researcher Affiliation | Academia | Feng Han1,2, Kai Chen1,2, Chao Gong1,2, Zhipeng Wei1,2, Jingjing Chen1,2*, Yu-Gang Jiang1,2 1Shanghai Key Lab of Intell. Info. Processing, School of Computer Science, Fudan University 2Shanghai Collaborative Innovation Center on Intelligent Visual Computing EMAIL, EMAIL |
| Pseudocode | No | The paper describes the methodology using prose and mathematical equations but does not contain a clearly labeled pseudocode block or algorithm. |
| Open Source Code | Yes | Code https://github.com/Maplebb/Du Mo |
| Open Datasets | Yes | Following ESD (Gandikota et al. 2023), we utilize the Inappropriate Image Prompts (I2P) dataset (Schramowski et al. 2023), which contains 4,703 toxic text prompts... With regards to the generation of non-target concepts, we employ COCO-30K, the validation set of MS-COCO (Lin et al. 2014) dataset. |
| Dataset Splits | Yes | With regards to the generation of non-target concepts, we employ COCO-30K, the validation set of MS-COCO (Lin et al. 2014) dataset. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, processor types, or memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper does not specify any software dependencies with version numbers (e.g., Python, PyTorch, CUDA versions) needed to replicate the experiments. |
| Experiment Setup | Yes | We set the threshold of the Nudenet to 0.6 to detect naked parts of images on I2P... Where ... η represents the erasure strength. ... L = Lera2 + λLpre where λ is the preserve scale of unrelated concepts. ... We equally split those 1000 denoising steps into 20 groups, and each group owns 13 parameters corresponding to the number of skip connection layers. ... We conduct a variety of experiments on SD v1.4 ... For each template, 5 images are generated with the seed 2024. ... The modulation factors in TLMO are initialized to 1 |