Optimization Guarantees for Square-Root Natural-Gradient Variational Inference
Authors: Navish Kumar, Thomas Möllenhoff, Mohammad Emtiyaz Khan, Aurelien Lucchi
TMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments demonstrate the effectiveness of natural gradient methods and highlight their advantages over algorithms that use Euclidean or Wasserstein geometries. ... We present empirical results showcasing the fast convergence of NGD, attributed to its Newton-like update. These results are illustrated in Figure 2 and Figure 3. |
| Researcher Affiliation | Academia | Navish Kumar EMAIL University of Basel, Basel, Switzerland Department of Mathematics and Computer Science Thomas Möllenhoff EMAIL RIKEN Center for AI Project, Tokyo, Japan Mohammad Emtiyaz Khan EMAIL RIKEN Center for AI Project, Tokyo, Japan Aurelien Lucchi EMAIL University of Basel, Basel, Switzerland Department of Mathematics and Computer Science |
| Pseudocode | Yes | Algorithm 1 Square-Root Variational Newton (SR-VN) |
| Open Source Code | No | The paper does not contain any explicit statements about releasing the code or provide a link to a code repository for the methodology described in this work. The provided URL in the paper is for datasets, not code. |
| Open Datasets | Yes | Datasets. We consider eight different LIBSVM datasets (Chang & Lin, 2011), consisting of five small and three large-scale datasets. The description of these datasets is provided in Table 2 of Appendix F. ... 1Available at https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/ |
| Dataset Splits | Yes | Here, we show results for two small-scale datasets (see Figure 2), namely Diabetes-scale (n = 768, d = 8, ntrain = 614) and Mushrooms (n = 8124, d = 112, ntrain = 64, 99). For large-scale datasets (see Figure 3), we show MNIST (n = 70, 000, d = 784, ntrain = 60, 000), Covtype-scale (n = 581, 012, d = 54, ntrain = 500, 000), and Phishing (n = 11, 055, d = 68, ntrain = 8, 844) datasets. ... Table 2: Dataset Statistics and Model Hyperparameters (includes N, d, Ntrain) |
| Hardware Specification | Yes | All experiments are performed on NVIDIA Ge Force RTX 3090 GPUs. |
| Software Dependencies | No | The paper mentions using 'modern automatic-differentiation methods' and 'LIBSVM datasets' but does not specify any software names with version numbers that would be required to reproduce the experiments. |
| Experiment Setup | Yes | For all experiments, we first use grid search to tune model hyper-parameters, where the search is performed in a specific range of values. The resultant values were then fixed during our experiments. The statistics of the datasets and the model hyper-parameters used are given in Table 2. |