Generative Models are Self-Watermarked: Declaring Model Authentication through Re-Generation
Authors: Aditya Desu, Xuanli He, Qiongkai Xu, Wei Lu
TMLR 2024 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 Experiments This section aims to demonstrate the efficacy of our re-generation framework of authorship authentication on generated text and image separately. For both Natural Language Generation (NLG) and Image Generation (IG) scenarios, in Sections 5.2 and 5.3, we first generate the initial intended data samples followed by several steps of paraphrasing (for NLG) or inpainting (for IG) as re-generation, detailed in Algorithm 1. Then, we test the properties of these samples by three series of experiments. 1. Distance Convergence: We verify the convergence of the distance between all one-step regenerations k [1..K]. 2. Discrepancy: We illustrate the discrepancy between the distances by the authentic generative models and the suspected other models for contrast. 3. Verification: We report the verification performance using precision and recall of the identified samples by the authentic models vs those from other contrasting models. |
| Researcher Affiliation | Collaboration | Aditya Desu EMAIL The University of Melbourne Xuanli He EMAIL University College London Qiongkai Xu EMAIL Macquarie University The University of Melbourne Wei Lu EMAIL Singapore University of Technology and Design Skywork AI |
| Pseudocode | Yes | Algorithm 1 Generation Algorithm for Stage I. ... Algorithm 2 Verification Algorithm for Stage II. |
| Open Source Code | No | No concrete access to the authors' own source code is provided. Footnote 7 links to a third-party library (Hugging Face Diffusers) used in their experiments, not the authors' implementation of their methodology: "We utilize the inpainting pipeline Stable Diffussion and Stable Diffusion XL provided by Hugging Face (von Platen et al., 2022)." |
| Open Datasets | Yes | For image generation, we sample 200 captions as prompts each from MS-COCO dataset (COCO) (Lin et al., 2014) and Polo Club Diffusion DB dataset (POLO) (Wang et al., 2022) ... All models are trained on a subset of the LAION-2B dataset (Schuhmann et al., 2022) consisting of CLIP-filtered image-text pairs. |
| Dataset Splits | No | The paper mentions sampling specific numbers of sentences/images from datasets, for example: "we sample 200 sentences from the in-house data as the starting point for each model" and "we sample 200 captions as prompts each from MS-COCO dataset... and Polo Club Diffusion DB dataset". However, it does not provide explicit training, validation, or test splits for these datasets, nor does it specify splitting methodologies for reproducibility. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., exact GPU/CPU models, processor types, or memory amounts) used for running its experiments. It only mentions general concepts like 'GPU mecha' in a prompt example or 'TPU' in a citation of prior work, not as their own experimental setup. |
| Software Dependencies | No | The paper mentions using 'Hugging Face Diffusers' for image generation pipelines in footnote 7, but it does not specify a version number for this library or any other software dependencies. The text states: "We utilize the inpainting pipeline Stable Diffussion and Stable Diffusion XL provided by Hugging Face (von Platen et al., 2022)." |
| Experiment Setup | Yes | To mitigate biases arising from varied sampling strategies, we set the temperature and top p to 0.7 and 0.95 for all NLG experiments. ... we fix δ at 0.05 for ensuing evaluations unless stated otherwise. ... selecting optimal parameters k = 5 and δ = 0.05 ... we first mask a sub-region, i.e., 1/N (N = 10 in our experiments) of its pixels with fixed positions ... split all possible mask positions into N (N = 8 in our experiments) non-overlapped sets |