Sparse Transfer Learning Accelerates and Enhances Certified Robustness: A Comprehensive Study
Authors: Zhangheng Li, Tianlong Chen, Linyi Li, Bo Li, Zhangyang Wang
AAAI 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical results demonstrate that our method accelerates the verification process for downstream tasks by as much as 70-80%, with only slight reductions in certified accuracy compared to dense parameter updates. We further validate that this performance improvement is even more pronounced in the few-shot transfer learning scenario. ... In this section, our objective is to address two primary inquiries via comprehensive experiments: (1) How effectively does the proposed method hasten the certified verification process and amplify the certified robustness for a downstream task under L2-norm input perturbations? (2) How do Diff Stab and Rig L contribute to enhancing the certified robustness performance in the context of our proposed sparse transfer learning and verification methodology? To address the posed questions, we carry out experiments in two distinct settings across two datasets: |
| Researcher Affiliation | Academia | Zhangheng Li1, Tianlong Chen1, Linyi Li2, Bo Li2,3, Zhangyang Wang1 1University of Texas at Austin 2University of Illinois Urbana-Champaign 3University of Chicago EMAIL, EMAIL, EMAIL, EMAIL, EMAIL |
| Pseudocode | No | The paper describes the methodology in narrative text and figures (e.g., Figure 1 illustrates the framework), but it does not contain any explicit pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain an explicit statement or a link indicating that source code for the described methodology is publicly available. |
| Open Datasets | Yes | We carry out experiments in two distinct settings across two datasets: CIFAR10 ... Celeb V-HQ Introduced in (Zhu et al. 2022), Celeb V-HQ is a contemporary benchmark tailored for multi-attribute classification tasks. |
| Dataset Splits | Yes | In the context of this dataset, we contemplate three distinct transfer settings: standard transfer, and a few-shot transfer, wherein the downstream tasks have access to merely 1% of randomly sampled data for their training. ... Our strategy then encompasses random sampling of 40 attributes for pre-training, with the remaining attributes earmarked for downstream transfer. |
| Hardware Specification | No | The paper mentions using Res Net-50, Res Net-18, and VGG-16 architectures, but it does not provide any specific details about the hardware (e.g., GPU models, CPU specifications) used for the experiments. |
| Software Dependencies | No | The paper does not specify any software dependencies or their version numbers (e.g., Python, PyTorch, TensorFlow versions) that would be needed to replicate the experiments. |
| Experiment Setup | Yes | Pertaining to the model architecture, unless stated otherwise, we consistently utilize Res Net-50 as the foundational network for our experiments. Only the fully connected layers of the network undergo reinitialization, with sparse transfer learning executed on the convolutional layers. We ve earmarked the perturbation radius of the input L2-norm ball at 0.25, considering a normalized image input. |