On the properties of variational approximations of Gibbs posteriors
Authors: Pierre Alquier, James Ridgway, Nicolas Chopin
JMLR 2016 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We specialize our results to several learning tasks (classification, ranking, matrix completion), discuss how to implement a variational approximation in each case, and illustrate the good properties of said approximation on real datasets. ... We now compare the numerical performance of the mean field and full covariance VB approximations to the Gibbs posterior (as approximated by SMC, see Section 3.1) for the classification of standard datasets; see Table 1. |
| Researcher Affiliation | Academia | Pierre Alquier EMAIL James Ridgway EMAIL Nicolas Chopin EMAIL ENSAE 3 Avenue Pierre Larousse 92245 MALAKOFF, FRANCE |
| Pseudocode | Yes | The pseudo-code below is given for an adaptive sequence of temperatures. Algorithm 1 Tempering SMC ... Algorithm 2 Systematic resampling ... Algorithm 3 Deterministic annealing ... Algorithm 4 Stochastic Gradient Descent |
| Open Source Code | Yes | We also provide a R package1, written in C++ to compute a Gaussian variational approximation in the case of the hinge risk. 1. PACVB package: https://cran.r-project.org/web/packages/PACVB/index.html |
| Open Datasets | Yes | The datasets are all available in the UCI repository3 except for the DNA dataset which is part of the R package mlbench by Leisch and Dimitriadou (2010). 3. https://archive.ics.uci.edu/ml/datasets.html |
| Dataset Splits | Yes | When no split between the training sample is provided we split the data in half. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU/CPU models or memory specifications used for running the experiments. It only mentions general aspects of numerical performance and convergence speed. |
| Software Dependencies | No | We also provide a R package1, written in C++ to compute a Gaussian variational approximation in the case of the hinge risk. While it mentions R and C++, no specific version numbers for these languages or any libraries are provided. |
| Experiment Setup | Yes | The hyperparameters are chosen by cross-validation. ... Stochastic VB with fixed temperature λ = 100 for Pima and λ = 1000 for adult. ... In all our experiments we take c = 1 and η = 0.9. |