Sequential Controlled Langevin Diffusions
Authors: Junhua Chen, Lorenz Richter, Julius Berner, Denis Blessing, Gerhard Neumann, anima anandkumar
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically demonstrate the performance of the proposed SCLD sampler on a wide variety of sampling benchmarks. Our SCLD method exhibits strong performance on both ELBO and Sinkhorn benchmarks (Tables 2 and 3). |
| Researcher Affiliation | Collaboration | 1University of Cambridge, 2Zuse Institute Berlin, 3dida Datenschmiede Gmb H, 4NVIDIA, 5Karlsruhe Institute of Technology, 6California Institute of Technology |
| Pseudocode | Yes | Algorithm 1 Sequential Controlled Langevin Diffusion (SCLD). See Algorithm 2 for details. ... For convenience, we state Algorithm 1 for a simplified, high-level overview of combining SMC with diffusion models and refer to Algorithm 2 in Appendix A.3 for a more detailed exposition. Further, we note that the suggested setting relates to the usual SMC algorithm (such as in Dai et al. (2022)) by taking a different forward transport step (where our Markov kernel is implemented by an SDE) and by adopting the weighting step (using the Radon-Nikodym derivative in place of the likelihood ratio). ... In Algorithm 2, we give a practical and detailed version of Algorithm 1. Algorithm 3 SCLD-Training ... Algorithm 4 SCLD-Buffer-Training ... Algorithm 5 Adaptive Multinomial Resampling |
| Open Source Code | Yes | Our code can be found at https://github.com/anonymous3141/SCLD. |
| Open Datasets | Yes | Examples from Bayesian statistics: The Seeds, Sonar, Credit, Brownian, and LGCP tasks. ... Synthetic targets: A 40-mode Gaussian mixture model in 50d (GMM40), a 32-mode Many-Well task (MW54) in 5d, the popular 10d Funnel benchmark, and a 50d Student mixture model (Mo S). ... The Robot1 and Robot4 tasks: Inspired by robotics control problems... Groundtruth samples are generated by long slice sampling runs (Neal, 2003) and taken from the repository of Arenz et al. (2020). |
| Dataset Splits | No | The paper describes using a fixed number of samples for evaluation (K = 2000 particles) and mentions that groundtruth samples are available for some tasks. It does not provide explicit train/test/validation splits with percentages or counts, as the task is sampling from a target density rather than traditional supervised learning with predefined data splits. |
| Hardware Specification | Yes | All experiments were performed on a single Nvidia RTX4090 GPU using the same settings as the main experiments. |
| Software Dependencies | No | We worked in the JAX framework and used jitting, discarding the first iteration (Bradbury et al., 2018). ... We use the Sinkhorn distance as implemented in Cuturi et al. (2022). ... We utilize the Adam optimizer for all methods that require learning. The paper mentions software tools like JAX, Sinkhorn distance implementation, and Adam optimizer but does not specify their version numbers. |
| Experiment Setup | Yes | We use batch size 2000 for training except for LGCP, where batch size 300 is used. We always evaluate with K = 2000 particles. ... We utilize the Adam optimizer for all methods that require learning. ... We select the number of training iterations such that all methods are given roughly the same number of target function evaluations (NFEs)... We use 40000 iterations for DDS and PIS... We sweep over σ in [0.1, 1, 10] for tasks where we have no information about the target. ... We set the minimum diffusion noise level to 0.01 for all tasks and methods... For all methods and tasks we perform grid searches over the maximum diffusion parameter. ... We sweep over the learning rate of the model in [10 3, 10 4, 10 5] and learning rate of the annealing schedule in [10 2, 10 3]. Tables 5 and 6 provide detailed hyperparameter choices for each task and method. |