Semi-Implicit Variational Inference via Score Matching
Authors: Longlin Yu, Cheng Zhang
ICLR 2023 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we compare SIVI-SM to ELBO-based methods including the original SIVI and UIVI on a range of inference tasks. We first show the effectiveness of our method and illustrate the role of the auxiliary network approximation fψ on several two-dimensional toy examples. The KL divergence from the target distributions to different variational approximations was also provided for direct comparison. We also compare the performance of SIVI-SM with both baseline methods on several Bayesian inference tasks, including a multidimensional Bayesian logistic regression problem and a high dimensional Bayesian multinomial logistic regression problem. |
| Researcher Affiliation | Academia | Longlin Yu School of Mathematical Sciences Peking University, Beijing, China EMAIL Cheng Zhang School of Mathematical Sciences and Center for Statistical Science Peking University, Beijing, China EMAIL |
| Pseudocode | Yes | Algorithm 1 SIVI-SM with multivariate Gaussian conditional layer |
| Open Source Code | No | The paper does not contain any statement about releasing source code or a link to a code repository. |
| Open Datasets | Yes | We consider the waveform3 dataset... 3https://archive.ics.uci.edu/ml/machine-learning-databases/waveform/. We used two data sets: MNIST4 and HAPT5. MNIST is a commonly used dataset in machine learning... 4http://yann.lecun.com/exdb/mnist/ 5http://archive.ics.uci.edu/ml/machine-learning-databases/00341/. Bayesian Neural Network on the UCI datasets. |
| Dataset Splits | Yes | The datasets are all randomly partitioned into 90% for training and 10% for testing. |
| Hardware Specification | Yes | The following table shows the run times of different methods per iteration on a RTX2080 GPU. |
| Software Dependencies | No | The paper states 'All experiments were implemented in Pytorch (Paszke et al., 2019).' but does not specify a version number for PyTorch or any other software. |
| Experiment Setup | Yes | If not otherwise specified, we use the Adam optimizer for training (Kingma & Ba, 2014). For SIVI-SM, we set the number of inner-loop gradient steps K = 1. For SIVI, we set L = 50 for the surrogate ELBO defined in Eq. 2. For UIVI, we used 10 iterations for every inner-loop HMC sampling. |