Regularizing Hard Examples Improves Adversarial Robustness
Authors: Hyungyu Lee, Saehyung Lee, Ho Bae, Sungroh Yoon
JMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct both theoretical and empirical analyses of this memorization phenomenon, showing that pruning hard examples in adversarial training can enhance the model s robustness. We assess our proposed methodology and find that DPLS could successfully leverage hard examples while circumventing the negative effect. Using the results of experiments on a variety of datasets and algorithms, we assess our proposed methodology and find that DPLS could successfully leverage hard examples while avoiding the negative effect. |
| Researcher Affiliation | Academia | Hyungyu Lee EMAIL Electrical and Computer Engineering Interdisciplinary Program in Artificial Intelligence Seoul National University, Seoul 08826, Republic of Korea Saehyung Lee EMAIL Electrical and Computer Engineering Interdisciplinary Program in Artificial Intelligence Seoul National University, Seoul 08826, Republic of Korea Ho Bae* EMAIL Department of Cyber Security Ewha Womans University, Seoul 03760, Republic of Korea Sungroh Yoon* EMAIL Electrical and Computer Engineering Interdisciplinary Program in Artificial Intelligence AIIS, ASRI, INMC, and ISRC, Seoul National University Seoul National University, Seoul 08826, Republic of Korea |
| Pseudocode | Yes | The procedure of DPLS is described in Algorithms 1 and 2. Algorithm 1 DPLS adversarial training. Algorithm 2 Calculation of DPLS. |
| Open Source Code | No | The paper does not explicitly state that the authors' implementation code for the described methodology (DPLS) is open-source or provide a direct link to a code repository for their work. The github link under the TRADES reference is for a third-party tool used by the authors, not their own source code. |
| Open Datasets | Yes | We conducted experiments on CIFAR-10, CIFAR-100 (Krizhevsky et al., 2009), SVHN (Netzer et al., 2011), and STL-10 (Coates et al., 2011) by employing PGD (Madry et al., 2017) and TRADES (Zhang et al., 2019) as the baseline adversarial training algorithms. |
| Dataset Splits | Yes | We select the top 10k hard and top 10k easy example subsets from the training dataset of CIFAR-10... We constituted datasets for both standard and adversarial (Madry et al., 2017) training by selecting 5k hard and 5k random examples from the training set... We trained several models by varying the adversarial budget ϵ = 8 (PGD), 4, 2, and 0 (STD), and the result is obtained after the learning rate decay epoch, which is the training point indicating that the model is sufficiently trained. |
| Hardware Specification | Yes | We used a single RTX 8000 GPU with CUDA11.6 and Cu DNN7.6.5 in our experiments. |
| Software Dependencies | Yes | We used a single RTX 8000 GPU with CUDA11.6 and Cu DNN7.6.5 in our experiments. |
| Experiment Setup | Yes | The learning rates for CIFAR-10, CIFAR-100, and STL-10 are set to 0.1 and 0.01 for SVHN, and the decay at 100 and 105 of the total training epoch 110 with decay factor 0.1 following Pang et al. (2021). We used stochastic gradient descent optimizer with the weight decay factor 5e-4 and the momentum 0.9. The upper bounds of adversarial perturbation were set to 0.031 (ϵ = 8), 0.0155 (ϵ = 4), and 0.00775 (ϵ = 2), and the step-size of training adversarial examples of each model were set to one fourth of the ℓ -bound of each model with 10 steps. |