Tracking the Copyright of Large Vision-Language Models through Parameter Learning Adversarial Images
Authors: Yubo Wang, Jianting Tang, Liu, Linli Xu
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments demonstrate that our method can more effectively identify the original copyright of fine-tuned models compared to baseline methods. Therefore, this work provides a powerful tool for tracking copyrights and detecting unlicensed usage of LVLMs. ... We report the TMRs of our proposed PLA and the baseline methods for copyright tracking on six fine-tuned models in Table 1. |
| Researcher Affiliation | Academia | Yubo Wang1,2, Jianting Tang1,2, Chaohu Liu1,2, Linli Xu1,2 1 University of Science and Technology of China 2 State Key Laboratory of Cognitive Intelligence EMAIL EMAIL |
| Pseudocode | Yes | Algorithm 1 PLA: Parameter Learning Attack |
| Open Source Code | No | The paper does not contain any explicit statement about making its source code publicly available, nor does it provide a link to a code repository. It mentions LLaVA-1.5 as an open-source model used in their work, but not their own implementation code. |
| Open Datasets | Yes | We initialize the trigger images using regular images randomly sampled from the Image Net 2012 validation set (Russakovsky et al., 2015). ... The datasets include the grounded VQA V7W (Zhu et al., 2016), text-related VQA ST-VQA and Text VQA (Biten et al., 2019; Singh et al., 2019), the artwork image VQA Painting Form (Bin et al., 2024), the mathematical VQA Math V360k (Shi et al., 2024), and the molecular graph VQA Ch EBI-20 (Edwards et al., 2021). |
| Dataset Splits | No | The paper describes sampling certain numbers of samples for fine-tuning specific datasets (e.g., 'sample 28k, 20k, and 50k samples for fine-tuning' or 'fine-tune with all the training data'), and randomly sampling 200 images for trigger initialization, but it does not specify explicit training, validation, and test splits for the experimental setup or evaluation. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used to run the experiments, such as GPU models, CPU types, or memory specifications. |
| Software Dependencies | No | The paper mentions algorithms and training configurations (e.g., 'PGD algorithm', 'optimizer Adam W', 'dtype bfloat16') but does not specify any software names with version numbers, such as Python, PyTorch, or CUDA versions. |
| Experiment Setup | Yes | For the adversarial attack, we employ the commonly used PGD algorithm (Madry, 2017) with 1000 iterations. The step size of trigger images α is set to 1/255. To enhance the concealment of trigger images, we set the perturbation size ϵ to 16/255. For model updates in PLA, we set the learning rate β to 1e-4 and the gradient clipping threshold to 5e-3. ... Table 5: Detailed configuration of full fine-tuning and Lo RA fine-tuning. Hyperparameter Full Fine-tuning Lo RA Fine-tuning optimizer Adam W Adam W learning rate 5e-5 2e-4 batch size 2 8 gradient accumulation 2 1 lr scheduler cosine cosine training epochs 3 3 dtype bfloat16 bfloat16 warmup steps 100 50 |