Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1]
Input Space Mode Connectivity in Deep Neural Networks
Authors: Jakub Vrabel, Ori Shem-ur, Yaron Oz, David Krueger
ICLR 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present theoretical and empirical evidence of its presence in the input space of deep networks, thereby highlighting the broader nature of the phenomenon. We observe that different input images with similar predictions are generally connected, and for trained models, the path tends to be simple, with only a small deviation from being a linear path. Our methodology utilizes real, interpolated, and synthetic inputs created using the input optimization technique for feature visualization. |
| Researcher Affiliation | Academia | Jakub Vrabel CEITEC, Brno University of Technology Ori Shem-Ur Tel Aviv University Yaron Oz Tel Aviv University David Krueger Mila, University of Montreal |
| Pseudocode | No | The paper describes methodologies and procedures in narrative text, for example in Section 3, "METHODOLOGY" and its subsections, but does not present any formal pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any explicit statements about releasing source code, nor does it provide any links to code repositories. |
| Open Datasets | Yes | First, we selected representative low-loss examples from the validation dataset, which have cross-entropy loss in the order of 1e-4 or lower. These examples were linearly interpolated with 1,000 uniformly distributed points. [...] We used Goog Le Net (Inception v1) (Szegedy et al., 2014) that was pretrained on the Image Net (Russakovsky et al., 2015) and is available from the Py Torch torchvision library (Paszke et al., 2019). [...] We implemented the algorithm to an existing benchmark (Harder et al., 2021) that uses VGG-16 (Simonyan & Zisserman, 2015) model and CIFAR10/100 datasets. |
| Dataset Splits | Yes | Seven unique pairs of images per class were selected from the validation subset of Image Net, encompassing 1,000 classes in total. [...] The attacks are applied on the CIFAR-10/100 test set and the VGG-16 NET. |
| Hardware Specification | Yes | Experiments have minimal compute requirements and were carried out on a regular PC (equipped with GTX 1650 4GB) or cloud services (T4 16GB). |
| Software Dependencies | No | The paper mentions using "Py Torch torchvision library (Paszke et al., 2019)" and "Adam optimizer (Kingma & Ba, 2017)", but does not specify version numbers for any software components. |
| Experiment Setup | Yes | The Adam optimizer (Kingma & Ba, 2017) was used with a learning rate of 0.005. We added regularization to the optimization with two additional terms: deviation penalty PMSE between B and B , and a high-frequency term Phf that penalizes changes in adjacent pixels (see definitions in App. B). The weights for these terms, λMSE = 0.1 for the deviation, and λhf ranging from 1e 8 to 5e 6 for the high-frequency term, were determined heuristically, varying by class and setup. [...] Adam (learning rate 0.005), 1024 iterations, cross-entropy loss with penalizations (λMSE = 0.1 for image deviation term PMSE and λhf = 1e 7 for high-freq. penalty PHF, see definitions in App. B). |