Prior Learning in Introspective VAEs
Authors: Ioannis Athanasiadis, Fredrik Lindsten, Michael Felsberg
TMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we perform a series of targeted experiments on a 2D density estimation benchmark and in an image generation setting comprised of the (F)-MNIST and CIFAR-10 datasets demonstrating the effect of prior learning in S-Intro VAE in generation and representation learning. [...] In this section we investigate the impact of learning the prior in S-Intro VAE. Our testbed consists of a 2D density estimation benchmark alongside three image datasets of varying complexity. [...] We evaluate the generation quality using the FID metric for samples generated from sampling from the prior and the aggregated posterior denoted as FID(GEN) and FID(REC) respectively. [...] The quality of the representations learned by the encoder was evaluated by fitting a linear SVM, similar to Kviman et al. (2023), using 2K-SVM and 10K-SVM iterations as well as utilizing a k-nearest neighbor classifier (k-NN) using 5-NN or 100-NN (Caron et al., 2021). |
| Researcher Affiliation | Academia | Ioannis Athanasiadis EMAIL Department of Electrical Engineering Linköping University Fredrik Lindsten EMAIL Department of Computer and Information Science Linköping University Michael Felsberg EMAIL Department of Electrical Engineering Linköping University |
| Pseudocode | Yes | In this section, we outline the implementation choices as well as the motivation behind them enabling prior learning in S-Intro VAE in a prior decoder cooperation manner. Pseudo-code for the prior learning in SIntro VAE is provided in Algorithm 1. |
| Open Source Code | No | The paper does not contain any explicit statement about making the code available for the work described, nor does it provide a link to a code repository. |
| Open Datasets | Yes | Finally, we perform a series of targeted experiments on a 2D density estimation benchmark and in an image generation setting comprised of the (F)-MNIST and CIFAR-10 datasets demonstrating the effect of prior learning in S-Intro VAE in generation and representation learning. [...] Our testbed consists of a 2D density estimation benchmark alongside three image datasets of varying complexity. |
| Dataset Splits | No | The paper mentions using well-known datasets like (F)-MNIST and CIFAR-10, which typically have standard splits. It also mentions evaluating representations by fitting a linear SVM using "2K-SVM and 10K-SVM iterations" and k-NN classifiers. However, it does not explicitly provide specific dataset split percentages, sample counts for training/validation/test sets for the main model training, or citations to predefined splits for the experimental setup. |
| Hardware Specification | No | The computational resources were provided by the National Academic Infrastructure for Supercomputing in Sweden (NAISS) at C3SE, partially funded by the Swedish Research Council through grant agreement no. 2022-06725. This statement mentions the resource provider but does not include specific hardware details such as GPU/CPU models, memory, or processor types. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers, such as programming language versions or library versions (e.g., Python 3.x, PyTorch 1.x). |
| Experiment Setup | Yes | In line with Daniel & Tamar (2021), we identified the optimal hyperparameters (i.e., βrec, βKL and βneg) by performing an extensive grid-search while we used α = 2 and γ = 1. [...] For our all our experiments we used K = 10. [...] Algorithm 1 Prior Learning in S-Intro VAE (Daniel & Tamar, 2021). The red-highlighted segments indicate the parts that differ from the standard Gaussian S-Intro VAE. The Lrec and the LKL refer to the reconstruction loss and the KL divergence between the posterior and the prior target respectively, whereas the Lsg KL is a modified KL divergence that applies the stop-gradient sg operator on the prior as target. Require: βrec, βKL, βneg, γ, η, rentropy, K |