Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1]
Denoised Smoothing: A Provable Defense for Pretrained Classifiers
Authors: Hadi Salman, Mingjie Sun, Greg Yang, Ashish Kapoor, J. Zico Kolter
NeurIPS 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | we demonstrate its effectiveness through extensive experimentation on Image Net and CIFAR-10. We verify the efficacy of our method through extensive experimentation on Image Net and CIFAR-10. We are able to convert pretrained Res Net-18/34/50 and Res Net-110, on CIFAR-10 and Image Net respectively, into certifiably robust models; our results are summarized in Tables 1 and 2 (details are in section 3). |
| Researcher Affiliation | Collaboration | Hadi Salman EMAIL Microsoft Research Mingjie Sun EMAIL CMU Greg Yang EMAIL Microsoft Research Ashish Kapoor EMAIL Microsoft Research J. Zico Kolter EMAIL CMU |
| Pseudocode | No | No structured pseudocode or algorithm blocks were found in the paper. |
| Open Source Code | Yes | Our code replicating all the experiments in the paper can be found at: https: //github.com/microsoft/denoised-smoothing1. |
| Open Datasets | Yes | extensive experimentation on Image Net and CIFAR-10. We are able to convert pretrained Res Net-18/34/50 and Res Net-110, on CIFAR-10 and Image Net respectively, into certifiably robust models |
| Dataset Splits | Yes | To assess the performance of our method on these APIs, we aggregate 100 random images from the Image Net validation set and certify their predictions across all four APIs. |
| Hardware Specification | No | No specific details about GPU or CPU models, memory, or cloud instance types used for experiments were provided in the main text. |
| Software Dependencies | No | The paper mentions software like 'Py Torch-pretrained Res Net' and denoisers like 'Dn CNN' and 'Mem Net', but does not provide specific version numbers for these or any other software dependencies. |
| Experiment Setup | Yes | In the following experiments, we only report the results for σ = 0.25,8 and we report the best curves over the denoiser architectures mentioned above. For more details on the architectures of the classifiers/denoisers, training/certification hyperparameters, etc., we refer the reader to Appendix A. |