On the Robustness of Neural Collapse and the Neural Collapse of Robustness
Authors: Jingtong Su, Ya Shi Zhang, Nikolaos Tsilivis, Julia Kempe
TMLR 2024 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through an extensive empirical investigation with computer vision datasets, we study the robustness of the formed simplices for several converged neural networks. Specifically, we find that gradient-based adversarial attacks with standard hyperparameters alter the feature representations, resulting in neither variability collapse nor simplex formation. |
| Researcher Affiliation | Academia | Jingtong Su EMAIL Center for Data Science New York University Ya Shi Zhang EMAIL Statistical Laboratory University of Cambridge Nikolaos Tsilivis EMAIL Center for Data Science New York University Julia Kempe EMAIL Center for Data Science and Courant Institute of Mathematical Sciences New York University |
| Pseudocode | No | The paper describes algorithms using mathematical equations (e.g., Equation 1 for PGD), but does not present them in clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available at https://github.com/Jingtong Su/ robust_neural_collapse. |
| Open Datasets | Yes | Datasets We consider image classification tasks on CIFAR-10, CIFAR-100 in our main text. Both datasets are balanced (in terms of images per class), so we comply with the original experimental setup of Papyan et al. (2020). We preprocess the images by subtracting their global (train) mean and dividing by the standard deviation. For the completeness of our experiments, we also consider a 10-class subset of Image Net: Image Nette2. We pick the 160px variant. The results are presented in Appendix F. |
| Dataset Splits | Yes | Datasets We consider image classification tasks on CIFAR-10, CIFAR-100 in our main text. Both datasets are balanced (in terms of images per class), so we comply with the original experimental setup of Papyan et al. (2020). ... For Image Nette, we pick the 160px variant. ... To fulfill the requirement of NC, we use the Center Crop of 160 to fix the train and test set. ... Previous research has observed that the neural collapse phenomenon under standard training does not hold on the test set (e.g., Hui et al. (2022).) We investigated the evolution of accuracy, loss, and neural collapse metrics for both clean and adversarially perturbed test set data. |
| Hardware Specification | No | This work was supported by the National Science Foundation under NSF Award 1922658, the Dean s Undergraduate Research Fund from the NYU College of Arts and Science, and in part through the NYU IT High Performance Computing resources, services, and staff expertise. |
| Software Dependencies | No | For ℓ and ℓ2 PGD attacks with ST and AT, we used the code from Rice et al. (2020a) 4. For TRADES, we adopted the original implementation5. |
| Experiment Setup | Yes | Algorithms We train the networks using stochastic gradient descent, either optimizing the cross entropy loss (standard training ST) or the worst case loss, bounded by either an ℓ2 or ℓ perturbation (adversarial training AT). For CIFAR-10/100 datasets, we adopt community-wide standard values for the perturbations following Rice et al. (2020a): for the ℓ adversary, we use radius ϵ = 8/255 and step size α = 2/255 in Equation 1. For the ℓ2 adversary, we use radius ϵ = 128/255 and step size α = 15/255. We perform 10 PGD iterations. All networks are being trained for 400 epochs in order to reach the terminal phase of training (post zero-error), with batch size 128 and initial learning rate 1e-1. We drop the learning rate by a factor of 0.1 at the 100th and again at the 150th epoch. We also consider the TRADES algorithm (Zhang et al., 2019) with Equation (4), setting β = 6 following Zhang et al. (2019). For Image Nette, we pick ℓ2 radius ϵ = 1536/255 and step size α = 360/255, and the same ℓ hyperparameters as for the CIFAR family. We perform 5 PGD iterations due to the larger image size. For full experimental details, please refer to Appendix B. |