Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1]
Diffusion Bridge Mixture Transports, Schrödinger Bridge Problems and Generative Modeling
Authors: Stefano Peluchetti
JMLR 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The theoretical findings are complemented by numerical experiments illustrating the competitive performance of the IDBM procedure. Recent advancements in generative modeling employ the time-reversal of a diffusion process to define a generative process that approximately transports a simple distribution to the data distribution. As an alternative, we propose utilizing the first iteration of the IDBM procedure as an approximation-free method for realizing this transport. This approach offers greater flexibility in selecting the generative process dynamics and exhibits accelerated training and superior sample quality over larger discretization intervals. In terms of implementation, the necessary modifications are minimally intrusive, being limited to the training loss definition. |
| Researcher Affiliation | Industry | Stefano Peluchetti EMAIL Cogent Labs 106 0032, Tokyo, Japan. |
| Pseudocode | Yes | Algorithm 1 IPF Algorithm 2 IDBM Algorithm 3 SGM training Algorithm 4 BDBM training |
| Open Source Code | Yes | Detailed settings of all hyperparameters defining this experiment, along with implementation details, are provided in the accompanying code implementation7. 7. https://github.com/stepelu/idbm-pytorch Py Torch implementations of SGM and BDBM losses are provided for a reference SDE described by (33) with Σ = I and for the regularizer λt = v(0, t) in Listings 1 and 2 (Appendix C). |
| Open Datasets | Yes | We consider the CIFAR-10 dataset, hence d = 28 28 3. As baseline modeling choice we consider the VE SDE parametrization from Song et al. (2021) for βt... Following Bortoli et al. (2021), we explore the scenario where the initial distribution Γ is represented by the MNIST dataset (DMNIST), whereas the terminal distribution Υ is derived from the first five lowercase and the first five uppercase characters, specifically, a,...,e,A,...,E, of the EMNIST dataset (DEMNIST). |
| Dataset Splits | Yes | After every 50, 000 SGD steps, we compute the FID between the test portion of the CIFAR-10 dataset, Dtest CIFAR 10, and the corresponding 5, 000 samples generated from the SGM and BDBM models. Test FID values are calculated by initializing X0 from Dtest EMNIST (Dtest MNIST) and subsequently sampling the discretized path Xt to obtain model samples that are approximately distributed as Dtest MNIST (Dtest EMNIST). |
| Hardware Specification | Yes | generating 5, 000 samples from either the SGM or the BDBM model using the Euler scheme with a discretization interval t = 10 4 necessitates approximately one day of computation on the NVIDIA RTX 3090 GPU utilized in this experiment. |
| Software Dependencies | No | The paper mentions 'Py Torch implementations' and 'official PyTorch implementation' but does not provide specific version numbers for PyTorch or any other software libraries. |
| Experiment Setup | Yes | We set σmax = 1 and keep Υ = Nd(0, σ2 max I) for the generative process s initial distribution, without making efforts to optimize these hyperparameters. Aside from the modifications to the loss function, no additional changes are introduced to the training code, and training proceeds as for the baseline SGM. ... CIFAR-10 training occurs over 1, 300, 000 SGD steps. We consider three different levels of regularization σ = 1.0, 0.5, 0.2, and compare the IDBM and DIPF procedures over 60 iterations, with each iteration comprising 5, 000 SGD steps. ... We utilize a lightweight configuration of the U-Net neural network architecture as proposed by Dhariwal and Nichol (2021). |