Accelerating Non-Conjugate Gaussian Processes By Trading Off Computation For Uncertainty
Authors: Lukas Tatzel, Jonathan Wenger, Frank Schneider, Philipp Hennig
TMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | As we demonstrate on large-scale classification problems, our method significantly accelerates posterior inference compared to competitive baselines by trading off reduced computation for increased uncertainty. (...) Experiments. We apply Iter NCGP to a Poisson regression problem to explore the trade-off between the number of (outer loop) mode-finding steps and (inner loop) solver iterations (Section 5.1). In Section 5.2, we demonstrate our algorithm s scalability and the impact of compression on performance. |
| Researcher Affiliation | Academia | Lukas Tatzel EMAIL University of Tübingen, Tübingen AI Center Jonathan Wenger EMAIL Columbia University Frank Schneider EMAIL University of Tübingen, Tübingen AI Center Philipp Hennig EMAIL University of Tübingen, Tübingen AI Center |
| Pseudocode | Yes | Algorithm 1: Iter NCGP Outer loop. (...) Algorithm 2: Iter NCGP Inner Loop: Iter GP with a Virtual Solver Run. (...) Algorithm 3: Recycling: Virtual Solver Run with Optional Compression. |
| Open Source Code | No | The paper mentions using 'GPy Torch (Gardner et al., 2018)' and 'Ke Ops (Charlier et al., 2021)' which are third-party libraries. There is no explicit statement or link provided in the paper for the authors' own implementation code for the methodology described. |
| Open Datasets | Yes | To demonstrate Iter NCGP s applicability to real-world data, we perform a similar experiment on MNIST (Lecun et al., 1998), see Appendix C.4 for details. |
| Dataset Splits | Yes | Data & Model. We generate N 10^5 data points from a Gaussian mixture model with C 10 classes. (...) For testing, N 10^4 data points are used (10^3 per class). (...) Data & Model. We use 20,000 training and 10,000 test images from the MNIST dataset and the softmax likelihood. |
| Hardware Specification | Yes | The benchmark is run on an NVIDIA A100 GPU. (...) All runs are performed on a single NVIDIA Ge Force RTX 2080 Ti 12 GB GPU. |
| Software Dependencies | No | The paper mentions software like 'GPy Torch', 'Ke Ops', 'Py Torch', 'Adam', and 'torchmetrics' but does not provide specific version numbers for any of them. For example: 'We use GPy Torch (Gardner et al., 2018) SVGP implementation and optimize the ELBO for 10^4 seconds using Adam with batch size 1024.' and 'We use Ke Ops (Charlier et al., 2021) and GPy Torch for fast kernel-matrix multiplies.'. |
| Experiment Setup | Yes | The learning rate α P t0.001, 0.01, 0.05u and the number of inducing points U P t1000, 2500, 5000, 10000u are tuned via grid search (...) optimize the ELBO for 10^4 seconds using Adam with batch size 1024 (...) Iter NCGP-CG is applied to the full training set with recycling and R P t8, 10u. The number of solver iterations is limited by j ď 5. (...) A fixed budget of 100 iterations is distributed uniformly over 5, 10, 20 or 100 outer loop steps (...) The convergence tolerance in Algorithm 1 is set to δ 0.001. |