ProFeAT: Projected Feature Adversarial Training for Self-Supervised Learning of Robust Representations
Authors: Sravanti Addepalli, Priyam Dey, Venkatesh Babu Radhakrishnan
TMLR 2024 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through extensive experiments on several benchmark datasets and models, we demonstrate significant improvements in both clean and robust accuracy when compared to existing SSL-AT methods, setting a new state-of-the-art. We further report on-par/ improved performance when compared to TRADES, a popular supervised-AT method. |
| Researcher Affiliation | Academia | Sravanti Addepalli Priyam Dey R. Venkatesh Babu Vision and AI Lab, Department of Computational and Data Sciences, IISc Bangalore |
| Pseudocode | No | The paper describes methods using mathematical equations and textual descriptions, for example, in Section 4 'Proposed Method' and its subsections. However, it does not contain any clearly labeled 'Pseudocode' or 'Algorithm' blocks, nor does it present structured steps formatted like code. |
| Open Source Code | Yes | Our code is available at: https://github.com/val-iisc/Pro Fe AT. |
| Open Datasets | Yes | Through extensive experiments on several benchmark datasets and models... on the standard benchmark datasets CIFAR-10 and CIFAR-100... on CIFAR-10 and CIFAR-100 datasets Krizhevsky et al. (2009)... We consider transfer from CIFAR-10/100 to STL-10 (Coates et al., 2011) and Caltech-101 (Li et al., 2022). |
| Dataset Splits | Yes | CIFAR-10 consists of 50,000 images in the training set and 10,000 images in the test set... We use a class-balanced validation split consisting of 1000 images from the train set and perform early-stopping during training based on the performance on the validation set. |
| Hardware Specification | Yes | The following Nvidia GPUs have been used for performing the experiments reported in this work V100, A100, and A6000. Each of the experiments are run either on a single GPU, or across 2 GPUs based on the complexity of the run and GPU availability. For 100 epochs of single-precision (FP32) training with a batch size of 256, the proposed approach takes 8 hours and 16GB of GPU memory on a single A100 GPU for Wide Res Net-34-10 model on CIFAR-100. |
| Software Dependencies | No | The paper mentions using the 'solo-learn' GitHub repository for Sim CLR SSL training and 'SGD optimizer' for various training phases. It also states 'Python 3.8, PyTorch 1.9, and CUDA 11.1' as an example of specific dependencies in the question context, but the paper itself does not explicitly list specific version numbers for Python, PyTorch, CUDA, or other key software libraries used for the implementation beyond the 'solo-learn' repository. |
| Experiment Setup | Yes | For the Sim CLR SSL training, we tune and use a learning rate of 1.5 with SGD optimizer, a cosine schedule with warmup, weight decay of 1e 5 and train the backbone for 1000 epochs... The self-supervised adversarial training... is performed for 100 epochs using SGD optimizer with a weight decay of 3e 4, cosine learning rate with 10 epochs of warm-up, and a maximum learning rate of 0.5... A 5-step PGD attack is used for attack generation during training with a step size of 2/255. We fix the value of β, the robustness-accuracy trade-off parameter (Ref: Equations (1) and (2) in the main paper) to 8 in all our experiments, unless specified otherwise. For 100 epochs of single-precision (FP32) training with a batch size of 256. |