Adapting to Evolving Adversaries with Regularized Continual Robust Training
Authors: Sihui Dai, Christian Cianfarani, Vikash Sehwag, Prateek Mittal, Arjun Bhagoji
ICML 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on 3 datasets (CIFAR-10, CIFAR-100, and Image Nette) and over 100 attack combinations demonstrate that the proposed regularization improves robust accuracy with little overhead in training time. Our key contributions are as follows: ... Empirical Validation on Sequentially Introduced Attacks ( 4.2). We conduct experiments on 2 sequences of 4 attacks across 3 datasets (CIFAR-10, CIFAR-100, and Imagenette). |
| Researcher Affiliation | Collaboration | 1Capital One 2Department of Electrical and Computer Engineering, Princeton University 3Department of Computer Science, University of Chicago 4Google Deepmind 5Centre for Machine Intelligence and Data Science, Indian Institute of Technology, Bombay. Correspondence to: Sihui Dai <EMAIL>, Christian Cianfarani <EMAIL>. |
| Pseudocode | No | The paper describes its methodology through prose and mathematical formulations within the main text and appendices, but it does not contain any clearly labeled 'Pseudocode' or 'Algorithm' blocks or figures. |
| Open Source Code | Yes | Our findings and open-source code1 lay the groundwork for the deployment of models robust to evolving attacks. 1Our code is available at: https://github.com/inspire-group/continual_robust_training/ |
| Open Datasets | Yes | We experiment with CIFAR-10, CIFAR-100 (Krizhevsky et al., 2009), and Image Nette (Howard), a 10-class subset of Image Net (Deng et al., 2009). |
| Dataset Splits | Yes | We experiment with CIFAR-10, CIFAR-100 (Krizhevsky et al., 2009), and Image Nette (Howard), a 10-class subset of Image Net (Deng et al., 2009)... We report accuracy on each attack, Union accuracy (overall accuracy when the worst case attack is chosen for each test example), Average accuracy (average over accuracy on each attack)... |
| Hardware Specification | No | The paper discusses experimental setup details such as training procedures, attack parameters, and model selection. However, it does not provide specific hardware details (e.g., GPU models, CPU types, or memory specifications) used for running the experiments. |
| Software Dependencies | No | The paper mentions using '10-step Projected Gradient Descent' and 'SGD with momentum of 0.9 and weight decay of 0.0005' but does not specify any software libraries (e.g., PyTorch, TensorFlow) along with their version numbers. |
| Experiment Setup | Yes | During training, we use 10-step Projected Gradient Descent (Madry et al., 2018) to generate adversarial examples. For the regularization terms (§ 3.3), VR and ALR use single step optimization to reduce time overhead, while UR and GR use σ = 2 and σ = 0.2, respectively... We train models for 100 epochs for initial training and 10 epochs for fine-tuning... For initial training, we start with a learning rate of 0.1 and then use the multistep learning rate scheduling proposed by Gowal et al. (2020); specifically, we scale the learning rate down by a factor of 10 halfway and 3/4 of the way through initial training or fine-tuning. For fine-tuning, we maintain a learning rate of 0.001. We train with SGD with momentum of 0.9 and weight decay of 0.0005. |