Bayesian Transferability Assessment for Spiking Neural Networks
Authors: Haiqing Hao, Wenhui Wang
TMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The experiment is conducted on both neuromorphic datasets and static datasets. For experiments on neuromorphic datasets, we use ES-Image Net dataset (Lin et al., 2021) for pre-training and DVS128 Gesture (DVS128), CIFAR10DVS (C10-DVS), N-Caltech101 (N-C101) and N-MNIST for fine-tuning, and for that on static datasets, the SNN models are pre-trained on Image Net (Deng et al., 2009) and fine-tuned on CIFAR10 (C10), CIFAR100 (C100), Caltech101 (C101) and MNIST. ... We validate our proposed method through experiments, and confirm that our Bayesian-based method outperforms information theory-based methods like NCE and LEEP for SNNs on neuromorphic datasets, and achieves comparable results on static datasets. |
| Researcher Affiliation | Academia | Haiqing Hao EMAIL Department of Precision Instrument Tsinghua University Wenhui Wang EMAIL Department of Precision Instrument Tsinghua University |
| Pseudocode | Yes | Algorithm 1 Algorithm of maximum evidence method with averaged feature |
| Open Source Code | Yes | Code is available at https://github.com/haohq19/meaf-snn. |
| Open Datasets | Yes | Neuromorphic datasets are vision datasets recorded with event cameras (Gallego et al., 2020), which are the commonly used benchmark for SNNs (He et al., 2020). As SNNs can also apply to conventional frame-based images (static datasets), we also validate our method on these datasets. For experiments on neuromorphic datasets, we use ES-Image Net dataset (Lin et al., 2021) for pre-training and DVS128 Gesture (DVS128), CIFAR10DVS (C10-DVS), N-Caltech101 (N-C101) and N-MNIST for fine-tuning, and for that on static datasets, the SNN models are pre-trained on Image Net (Deng et al., 2009) and fine-tuned on CIFAR10 (C10), CIFAR100 (C100), Caltech101 (C101) and MNIST. |
| Dataset Splits | Yes | The test set accuracy of the best model on the validation set is used as the ground truth of transferability. ... Note that ranking models with the maximum model evidence does not require the involvement of the validation or test set. ... On the DVS128 Gesture dataset, the Kendall coefficient is 0.56, which is still higher than LEEP (0.2) and NCE (0.16). We believe that the drop of coefficient on DVS128 Gesture dataset is because the DVS128 Gesture is a small dataset, with only 1176 samples in the training set (Amir et al., 2017). |
| Hardware Specification | Yes | For experiments on neuromorphic datasets, models are pre-trained with Adam optimizer (Kingma & Ba, 2014) with momentum 0.9 and 0.999 for 10 epochs with 8 RTX3090 GPUs. |
| Software Dependencies | No | The paper mentions software like "Adam optimizer (Kingma & Ba, 2014)" and "Torch Vision (maintainers & contributors, 2016)" but does not provide specific version numbers for these or other software dependencies used in their implementation. |
| Experiment Setup | Yes | For experiments on neuromorphic datasets, models are pre-trained with Adam optimizer (Kingma & Ba, 2014) with momentum 0.9 and 0.999 for 10 epochs with 8 RTX3090 GPUs. We use a step learning rate scheduler with an initial learning rate of 0.01, step size 3, and γ = 0.3. For experiments on static datasets, we use PTMs from Fang et al. (2021). ... On neuromorphic datasets, each model is re-trained for 200 epochs. The hyper-parameters, learning rate, and weight decay, are determined by grid-searching from 1e-1 to 1e-3 and 1e-5 to 1e-8 respectively. On static datasets, each model is re-trained for 100 epochs. The same hyper-parameters are determined by grid-searching from 1e-1 to 1e-3 and 1e-5 to 1e-7 respectively. |