Fourier Sensitivity and Regularization of Computer Vision Models
Authors: Kiran Krishnamachari, See-Kiong Ng, Chuan-Sheng Foo
TMLR 2022 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show in extensive empirical evaluations that Fourier-regularization can indeed modify frequency characteristics of computer vision models, and can improve the generalization performance of models on o.o.d. datasets where the Fourier-statistics are shifted. ... 4 Experiments We first study below the Fourier-sensitivity of various architectures and training methods across datasets (Section 4.2). We found that both training method and architecture can have a significant impact on Fourier-sensitivity. We then identify an interesting connection between adversarial attacks and Fourier-sensitivity. Further, we study the effects of Fourier-regularization on representation learning (Section 4.3) as well as real o.o.d. benchmarks (Section 4.4). |
| Researcher Affiliation | Academia | Kiran Krishnamachari EMAIL Institute for Infocomm Research, A*STAR, Singapore School of Computing, National University of Singapore, Singapore; See-Kiong Ng EMAIL Institute of Data Science, National University of Singapore, Singapore School of Computing, National University of Singapore, Singapore; Chuan-Sheng Foo EMAIL Institute for Infocomm Research, A*STAR, Singapore Centre for Frontier AI Research, A*STAR, Singapore |
| Pseudocode | Yes | Algorithm 1 Fourier-sensitivity of a model Input: Labeled samples L = {(xi, yi)}n i=1; a model f with trained parameters θ Output: Estimated Fourier-sensitivity of model, f SF S( ; θ) for i = 1 to n do Compute loss LCE(f(xi), yi) {forward pass} Backpropagate LCE to obtain LCE xi {input-gradient, averaged across color channels} LCE xf i = F( LCE xi ) {unitary 2D DFT of input-gradient} f SF S(xi, yi) = [Pk; for k=1 to N/ 2] {see Equation 1; excludes DC component} end for f SF S( ; θ) = 1 n Pn i=1 f SF S(xi, yi) {estimated Fourier sensitivity of model} |
| Open Source Code | No | The paper mentions using the 'robustness (Engstrom et al., 2019) library for training' and provides a URL for it. However, it does not explicitly state that the authors' own code for the methodology described in the paper is released or provide a link to it. |
| Open Datasets | Yes | On CIFAR10 and CIFAR100 (Krizhevsky & Hinton, 2009), we trained all models for 150 epochs... On SVHN (Netzer et al., 2011), we trained models for 40 epochs... We computed the Fourier-sensitivity of pre-trained Image Net architectures obtained from Py Torch Image Models (Wightman, 2019). |
| Dataset Splits | Yes | On CIFAR10 and CIFAR100 (Krizhevsky & Hinton, 2009), we trained all models for 150 epochs... On SVHN (Netzer et al., 2011), we trained models for 40 epochs... We computed the Fourier-sensitivity of pre-trained Image Net architectures obtained from Py Torch Image Models (Wightman, 2019). ... We added noise at all frequencies to the respective test sets of SVHN and CIFAR10 to evaluate the sensitivity of models. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU models, CPU types, or memory used for running the experiments. It only describes the training parameters and software libraries used. |
| Software Dependencies | No | The paper mentions 'Py Torch Image Models (Wightman, 2019)' and the 'robustness (Engstrom et al., 2019) library' but does not specify their version numbers. |
| Experiment Setup | Yes | On CIFAR10 and CIFAR100 (Krizhevsky & Hinton, 2009), we trained all models for 150 epochs using stochastic gradient descent (SGD) with momentum (0.9), an initial learning rate of 0.1 decayed by a factor of 10 every 50 epochs, weight decay parameter equal to 5 10 4 and batch size equal to 128. On SVHN (Netzer et al., 2011), we trained models for 40 epochs using Nesterov momentum with an initial learning rate of 0.01 and momentum parameter 0.9. The training batch size was 128, L2 regularization parameter was 5 10 4 and learning rate was decayed at epochs 15 and 30 by a factor of 10. The following standard data augmentations random-crop, random-horizontal-flip, random-rotation, and color-jitter were used during training. ... We trained Fourier-regularized models using λSFS = 0.5... For adversarial training (AT), we used standard PGD ℓ2 attacks (ϵ = 1 for CIFAR10/CIFAR100 and ϵ = 3 for Image Net-subset, attack-steps = 7 attack-lr = ϵ/7). For Gaussian noise training, we added i.i.d. Gaussian noise drawn from N(0, σ2) to each pixel during training (σ = 0.1). |