Towards Adversarially Robust Dataset Distillation by Curvature Regularization
Authors: Eric Xue, Yijiang Li, Haoyang Liu, Peiran Wang, Yifan Shen, Haohan Wang
AAAI 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive empirical experiments suggest that our method not only outperforms standard adversarial training on both accuracy and robustness with less computation overhead but is also capable of generating robust distilled datasets that can withstand various adversarial attacks. Detailed evaluation of GUARD to demonstrate its effectiveness across multiple aspects |
| Researcher Affiliation | Academia | 1Department of Computer Science, University of Toronto 2Electrical and Computer Engineering, University of California San Diego 3School of Information Sciences, University of Illinois Urbana-Champaign 4Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign EMAIL, EMAIL, EMAIL, EMAIL, |
| Pseudocode | No | The paper describes the proposed method GUARD and its formulation using mathematical equations and prose (e.g., Equations 1, 4, 5, 6, 7, 8, 11), but does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain an explicit statement about releasing the source code for the methodology described, nor does it provide a link to a code repository. |
| Open Datasets | Yes | For a systematic evaluation of our method, we investigate the top-1 classification accuracy of models trained on data distilled from three commonly-used datasets in this domain: Image Nette (Howard 2018), Tiny Image Net (Le and Yang 2015), and Image Net (Deng et al. 2009). |
| Dataset Splits | Yes | For a systematic evaluation of our method, we investigate the top-1 classification accuracy of models trained on data distilled from three commonly-used datasets in this domain: Image Nette (Howard 2018), Tiny Image Net (Le and Yang 2015), and Image Net (Deng et al. 2009). ... In the evaluation phase, we train a Res Net18 model on the distilled dataset for 300 epochs, before assessing it on the test set of the original dataset. |
| Hardware Specification | Yes | Experiments are performed on one NVIDIA A100 80GB PCIe GPU with batch size 32. We measure 5 times per iteration training time and report the average and standard deviation. |
| Software Dependencies | No | The paper mentions using ResNet18 architecture and various adversarial attack methods, but does not specify any software names with version numbers for implementation (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | During the squeeze step of the distillation process, we train the model on the original dataset over 50 epochs using a learning rate of 0.025. Based on preliminary experiments, we determined that the settings h = 3 and ΜΈ = 100 provide an optimal configuration for our regularizer. In the recover step, we perform 2000 iterations to synthesize the images and run 300 epochs to generate the soft labels to obtain the full distilled dataset. In the evaluation phase, we train a Res Net18 model on the distilled dataset for 300 epochs, before assessing it on the test set of the original dataset. |