Zero-cost Proxy for Adversarial Robustness Evaluation
Authors: Yuqi Feng, Yuwei Ou, Jiahao Fan, Yanan Sun
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results show that the proposed zero-cost proxy can bring more than 20 speedup compared with the state-of-the-art robust NAS methods, while the searched architecture has superior robustness and transferability under white-box and black-box attacks. |
| Researcher Affiliation | Academia | Yuqi Feng, Yuwei Ou, Jiahao Fan, Yanan Sun College of Computer Science, Sichuan University EMAIL; EMAIL; EMAIL |
| Pseudocode | No | The paper describes the methodology using mathematical formulations and descriptive text, but does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our source code is available at https://github.com/fyqsama/Robust_ZCP. |
| Open Datasets | Yes | Benchmark Datasets Following the conventions of the robust NAS community (Guo et al., 2020; Mok et al., 2021; Ou et al., 2024), CIFAR-10 (Krizhevsky & Hinton, 2009), SVHN (Netzer et al., 2011), Tiny-Image Net-200 (Le & Yang, 2015), and Image Net (Deng et al., 2009) are used as benchmark datasets. Moreover, a novel dataset named Tiny-Robust Bench is open sourced to promote the development of this field. NAS-Bench-201-R (Jung et al., 2023): NAS-Bench-201-R is the first to test all pretrained NAS-Bench-201 models (containing 6,466 unique architectures) under various adversarial attacks, i.e., FGSM, PGD, APGD, and Square, with various attack parameters. |
| Dataset Splits | No | The paper refers to using benchmark datasets like CIFAR-10, SVHN, Tiny-Image Net-200, and Image Net, and mentions following 'conventions of the robust NAS community' for adversarial training settings. It also describes creating a 'Tiny-Robust Bench' dataset. However, it does not explicitly provide the specific training, validation, or test dataset splits used for these datasets within the text. |
| Hardware Specification | Yes | The search cost is measured by GPU days (number of GPUs used total running time (days)) using the NVIDIA RTX 2080Ti GPU. |
| Software Dependencies | No | The paper mentions optimizers like SGD and refers to PGD for adversarial training but does not provide specific software library names or version numbers (e.g., PyTorch, TensorFlow) that would be needed to replicate the experimental environment. |
| Experiment Setup | Yes | Neural architectures are stacked by 20 cells, with an initial channel number of 36. After the search, the best architecture is adversarially trained. Following advanced settings in the robust NAS community (Mok et al., 2021; Ou et al., 2024), the adversarial training is performed using a seven-step PGD with a step size of 0.01 and a total perturbation of 8/255. SGD is used to optimize networks, with the momentum of 0.9 and the weight decay of 1 10 4. The learning rate is set to 0.1 initially, and decayed by a factor of 0.1 at the 100-th epoch. The batch size is set to 64. Table 13: The training details for the architectures in Tiny-Robust Bench. Items Values Total Training Epoch 105 Initial Learning Rate 0.1 Learning Rate Decay Policy Stepped Decent Learning Rate Decent Factor 0.1 The Index of Epoch for Learning Rate Decent 99 Momentum 0.9 Weight Decay 0.0001 Adversarial Loss PGD Perturbation Rate 8/255 Number of Steps 7 Step Size 0.01 |