Asymptotic Unbiased Sample Sampling to Speed Up Sharpness-Aware Minimization
Authors: Jiaxin Deng, Junbiao Pang, Baochang Zhang, Guodong Guo
AAAI 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | On CIFAR-10/100 and Tiny Image Net, AUSAM achieves results comparable to SAM while providing a speedup of over 70%. By adjusting hyperparameters, AUSAM can match the speed of the base optimizer while significantly surpassing the base optimizer s performance. Additionally, AUSAM accelerates optimization in human pose estimation and model quantization without sacrificing performance, demonstrating its broad practicality. |
| Researcher Affiliation | Academia | 1School of Information Science and Technology, Beijing University of Technology, Beijing, China 2Hangzhou Research Institute, School of Artificial Intelligence, Beihang University, China 3Nanchang Institute of Technology, Nanchang, China 4Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, China |
| Pseudocode | Yes | Algorithm 1 shows the overall proposed algorithm. |
| Open Source Code | No | The paper does not contain any explicit statements about code availability or links to a code repository. |
| Open Datasets | Yes | We conduct experiments on the CIFAR-10, CIFAR-100 (Krizhevsky, Hinton et al. 2009) and Tiny-Image Net (Le and Yang 2015) image classification benchmark datasets. We also conducted experiments on the MPII dataset (Andriluka et al. 2014). |
| Dataset Splits | Yes | We conduct experiments on the CIFAR-10, CIFAR-100 (Krizhevsky, Hinton et al. 2009) and Tiny-Image Net (Le and Yang 2015) image classification benchmark datasets. We use SGD, SAM, and AUSAM as optimizers for the Sim CC, and conducted experiments on the MPII dataset (Andriluka et al. 2014). For Tiny-Image Net, we also train Res Net-18, Res Net-50 and Mobile Netv1 (Howard et al. 2017) for 200 epochs using a batch size of 128 with cutout and cosine learning rate decay. |
| Hardware Specification | Yes | We implement AUSAM in Pytorch and train models on a single NVIDIA Ge Force RTX 3090. |
| Software Dependencies | No | The paper mentions 'Pytorch' but does not specify a version number or other key software dependencies with their versions. |
| Experiment Setup | Yes | We trained all the models for 200 epochs with a batch size of 128, employing cutout regularization (De Vries and Taylor 2017) and cosine learning rate decay (Loshchilov and Hutter 2016) for all methods. For the proposed method, we set smin = 0.1, smax = 0.5 for Res Net-18 and Wide Res Net-28-10, smax = 1 for Pyramid Net-110. In the experiments, both SAM and AUSAM use the Adam optimizer as their base optimizer, with a batch size set to 64. |