ASTrA: Adversarial Self-supervised Training with Adaptive-Attacks
Authors: Prakash Chandra Chhipa, Gautam Vashishtha, Jithamanyu Settur, Rajkumar Saini, Mubarak Shah, Marcus Liwicki
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | ASTr A achieves state-of-the-art results on CIFAR10, CIFAR100, and STL10 while integrating seamlessly as a plug-and-play module for other self-AT methods. ASTr A shows scalability to larger datasets, demonstrates strong semi-supervised performance, and is resilient to robust overfitting, backed by explainability analysis on optimal attack strategies. Project page for source code and other details at https://prakashchhipa.github.io/projects/ASTr A. |
| Researcher Affiliation | Academia | 1Lule a Tekniska Universitet, Sweden, 2Indian Institute of Technology, Gandhinagar, India 3Indian Institute of Technology, Madras, India, 4University of Central Florida, USA |
| Pseudocode | Yes | A.2 ASTRA ALGORITHM Below are the details of the proposed ASTr A algorithm: Algorithm 1 ASTr A Algorithm Require: Training dataset D, target model fw, strategy model sθ 1: for each epoch do 2: for each batch B in D do 3: x augmented views of images in B 4: if update interval reached then 5: Set fw to eval mode, sθ to train mode 6: a sθ(x) Sample attack parameters 7: xadv PGD(fw, x, a) Generate adversarial examples 8: r Compute Reward(fw, x, xadv) 9: REINFORCEUPDATE(sθ, a, r, x) 10: end if 11: Set fw to train mode, sθ to eval mode 12: a sθ(x) Select attack parameters 13: xadv PGD(fw, x, a) Generate adversarial examples 14: z fw(x, normal ) Clean features 15: zadv fw(xadv, pgd ) Adversarial features 16: Lclean NT-Xent(z) Contrastive loss on clean samples 17: Ladv NT-Xent(zadv) Contrastive loss on adversarial samples 18: Lsim NT-Xent([z, zadv]) Similarity loss 19: L (Lclean + Ladv)/2 + λLsim 20: Compute gradients of L with respect to fw 21: Update fw parameters using computed gradients 22: end for 23: end for |
| Open Source Code | Yes | Project page for source code and other details at https://prakashchhipa.github.io/projects/ASTr A. The source code of ASTr A is made available at https://prakashchhipa.github.io/ projects/ASTr A and results can be reproduced with Py Torch 2.0 on CUDA 12.x version. |
| Open Datasets | Yes | Our framework outperforms existing self-AT methods in adversarial robustness across multiple datasets, including CIFAR10, CIFAR100, and STL100, and scales effectively to larger datasets such as Image Net-100. |
| Dataset Splits | Yes | We evaluate ASTr A on the benchmarks CIFAR10, CIFAR100 Krizhevsky et al. (2009), and STL10 Coates et al. (2011), comparing against existing self-AT methods: Ro CL Kim et al. (2020), ACL Jiang et al. (2020), Adv CL Fan et al. (2021), De ACL Zhang et al. (2022), DYNACL Luo et al. (2023), and DYNACL-AIR Xu et al. (2024). Additionally, we assess the scalability of ASTr A on the Image Net-100 Tian et al. (2020). All the methods are evaluated with Res Net18 under the same condition following Jiang et al. (2020). |
| Hardware Specification | Yes | Training conducted on single H100 GPU. |
| Software Dependencies | Yes | The source code of ASTr A is made available at https://prakashchhipa.github.io/ projects/ASTr A and results can be reproduced with Py Torch 2.0 on CUDA 12.x version. |
| Experiment Setup | Yes | Pretraining. We use Res Net-18 He et al. (2016) as the target network, incorporating a mixed contrastive loss term with a weighting parameter γ = 0.5, following the protocol from Jiang et al. (2020). Res Net-18 is also used as the adaptive strategy network, with a learning rate of 0.1, LARS optimizer, step sizes ranging from 1 to 6, attack iterations between 3 and 14, and a perturbation range of 3 to 15. Reward weights α and β are both set to 1.0 for adversarial and clean losses, respectively. In Res Net, we use bottleneck projector head of size 2048x512, performance comparison with ACL (Jiang et al. (2020) using ACL projector head is in sec. A.4.4 in appendix. We set β to 0.5 for mixed contrastive loss term. ASTr A++ is longer pretraining variant of ASTr A with 2000 epochs. |