Standard-Deviation-Inspired Regularization for Improving Adversarial Robustness
Authors: Olukorede Fakorede, Modeste Atsague, Jin Tian
TMLR 2024 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we conduct an extensive evaluation of the proposed method. To assess its versatility, we test it on various datasets, including CIFAR-10 (Krizhevsky et al., 2009), CIFAR-100 (Krizhevsky et al., 2009), SVHN (Netzer et al., 2011), and Tiny Image Net Deng et al. (2009). We apply simple data augmentations, such as 4-pixel padding with 32 32 random crop and random horizontal flip, to each of the datasets. Additionally, we employ Res Net-18 (He et al., 2016) and Wide Res Net-34-10 (He et al., 2016) as the backbone models. |
| Researcher Affiliation | Academia | Olukorede Fakorede EMAIL, EMAIL Department of Computer Science Iowa State University, Ames, Iowa, USA; Modeste Atsague EMAIL Department of Computer Science Iowa State University, Ames, Iowa,USA; Jin Tian EMAIL Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi, United Arab Emirates |
| Pseudocode | Yes | Algorithm 1 AT-SDI Algorithm. Input: a neural network model with the parameters θ, step size κ, T PGD steps, a training dataset D of size n, |C| is the number of classes, and hyperparameter β. Output: a robust model with parameters θ; Algorithm 2 SDI-PGD Algorithm. Input: a neural network model with the parameters θ, step size κ, natural examples xi in a labelled dataset D of size n and |C| is the number of classes. Output: Adversarial examples x i |
| Open Source Code | No | The paper does not provide an explicit statement about open-sourcing its code or a link to a code repository. |
| Open Datasets | Yes | To assess its versatility, we test it on various datasets, including CIFAR-10 (Krizhevsky et al., 2009), CIFAR-100 (Krizhevsky et al., 2009), SVHN (Netzer et al., 2011), and Tiny Image Net Deng et al. (2009). |
| Dataset Splits | No | We train the backbone networks using mini-batch gradient descent for 110 epochs, with a momentum of 0.9 and a batch size of 128. For training CIFAR-10, we used a weight decay of 5e-4, and for CIFAR-100, SVHN, and Tiny Image Net, we used a weight decay of 3.5e-3. The initial learning rate was set to 0.1 (0.01 for CIFAR-100, SVHN, and Tiny Image Net), and it was divided by 10 at the 75th epoch and then again at the 90th epoch. The hyperparameters are tuned using a validation set. |
| Hardware Specification | Yes | We conducted all experiments using a single core of an AMD EPYC 7513 processor, an Nvidia A100 SXM4 80 GB GPU, and 128 GB of RAM. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers. |
| Experiment Setup | Yes | We train the backbone networks using mini-batch gradient descent for 110 epochs, with a momentum of 0.9 and a batch size of 128. For training CIFAR-10, we used a weight decay of 5e-4, and for CIFAR-100, SVHN, and Tiny Image Net, we used a weight decay of 3.5e-3. The initial learning rate was set to 0.1 (0.01 for CIFAR-100, SVHN, and Tiny Image Net), and it was divided by 10 at the 75th epoch and then again at the 90th epoch. |