Long-time asymptotics of noisy SVGD outside the population limit
Authors: Victor Priser, PASCAL BIANCHI, Adil Salim
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We study the long-time asymptotic behavior of a noisy variant of SVGD. First, we establish that the limit set of noisy SVGD for large k is well-defined. We then characterize this limit set, showing that it approaches the target distribution as n increases. In particular, noisy SVGD avoids the variance collapse observed for SVGD. Our approach involves demonstrating that the trajectories of noisy SVGD closely resemble those described by a Mc Kean-Vlasov process. Section 6: NSVGD AVOIDS THE PARTICLES COLLAPSE Fig. 1 (see Appendix for larger figures) reproduces an experiment from Ba et al. (2021) on the variance collapse of SVGD. We added our algorithm, NSVGD, to the plot. In Fig. 2, we show that the collapse occurs even when the number of particles n is large compared to the dimension d. |
| Researcher Affiliation | Collaboration | V. Priser Télécom Paris P. Bianchi Télécom Paris A. Salim Microsoft Research |
| Pseudocode | Yes | Algorithm 1 Noisy Stein Variational Gradient Descent (NSVGD) Initialization: generate n particles (X1,n 0 , . . . , Xn,n 0 ) for k = 0, 1, 2, . . . do |
| Open Source Code | Yes | Our Python script is available in the Supplementary Material and Fig. 1 and 2 are available in the Appendix in a larger format. |
| Open Datasets | Yes | We consider the task of sampling from a standard Gaussian with NSVGD and SVGD. We consider the Neal funnel distribution (Neal, 2003), defined as: π((x1, x2)) = N(x1; 0, 3)N(x2; 0, ex1), |
| Dataset Splits | No | The experiments involve sampling from standard or theoretical distributions (standard Gaussian, Neal funnel distribution) which are not typically subjected to train/test/validation splits in the context of this paper. The paper does not mention any dataset splits. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used to run the experiments, such as GPU or CPU models. |
| Software Dependencies | No | The paper mentions 'Our Python script is available in the Supplementary Material' but does not specify any software versions for Python or any libraries used. |
| Experiment Setup | Yes | The particles are initialized randomly from a standard Gaussian and the step size is set to γk = 10/k. We simulate NSVGD until convergence (i.e., after a large number k = 200 of iterations) for different values of the dimension d, the number of particles n, and the regularization parameter λ. We consider n = 100 particles, γk = 0.1 and d = 2. |