Generalization Bounds with Logarithmic Negative-Sample Dependence for Adversarial Contrastive Learning
Authors: Naghmeh Ghanooni, Waleed Mustafa, Yunwen Lei, Anthony Widjaja Lin, Marius Kloft
TMLR 2024 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our theoretical results are supported by experiments on real-world datasets. Empirical evaluations are provided in Section 6. 6 Experiments We evaluate our theoretical results to two widely used benchmark datasets from the image domain: CIFAR-10 and CIFAR-100 (Krizhevsky, 2009). |
| Researcher Affiliation | Academia | Naghmeh Ghanooni EMAIL Department of Computer Science RPTU, Kaiserslautern, Germany Waleed Mustafa EMAIL Department of Computer Science RPTU, Kaiserslautern, Germany Yunwen Lei EMAIL Department of Mathematics University of Hong Kong, Hong Kong, China Anthony Widjaja Lin EMAIL MPI-SWS RPTU, Kaiserslautern, Germany Marius Kloft EMAIL Department of Computer Science RPTU, Kaiserslautern, Germany |
| Pseudocode | No | The paper describes theoretical framework and experimental results but does not contain any clearly labeled pseudocode or algorithm blocks. Methods are described textually and mathematically. |
| Open Source Code | No | The paper does not contain any explicit statements or links regarding the release of source code for the methodology described. |
| Open Datasets | Yes | We evaluate our theoretical results to two widely used benchmark datasets from the image domain: CIFAR-10 and CIFAR-100 (Krizhevsky, 2009). CIFAR-10 and CIFAR-100 consist of 50,000 training images and 10,000 testing images, organized into 10 and 100 classes, respectively. |
| Dataset Splits | Yes | CIFAR-10 and CIFAR-100 consist of 50,000 training images and 10,000 testing images, organized into 10 and 100 classes, respectively. |
| Hardware Specification | No | The paper mentions that models are trained but does not provide any specific details about the hardware used (e.g., GPU models, CPU types, or memory specifications). |
| Software Dependencies | No | Both models are trained using the Adam optimizer, with a learning rate of 1e-3. To generate adversarial perturbations, we employ the ℓ PGD algorithm (Madry, 2017) with a step size of ϵ/255. The paper mentions software components like 'Adam optimizer' and 'PGD algorithm' but does not provide specific version numbers for any software libraries or frameworks (e.g., PyTorch, TensorFlow, Python version). |
| Experiment Setup | Yes | Both models are trained using the Adam optimizer, with a learning rate of 1e-3. To generate adversarial perturbations, we employ the ℓ PGD algorithm (Madry, 2017) with a step size of ϵ/255, where ϵ represents the maximum allowable perturbation. We calculate the generalization error for different values of ϵ = {2/255, 4/255, 8/255, 16/255, 32/255, 64/255, 128/255}, with varying negative samples, K = {63, 127, 511, 1023}. |