Zero-Shot Conditioning of Score-Based Diffusion Models by Neuro-Symbolic Constraints
Authors: Davide Scassola, Sebastiano Saccani, Ginevra Carbone, Luca Bortolussi
AAAI 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we show the effectiveness of our approach in approximating conditional distributions for various types of constraints and data: tabular data, images and time series. We tested our method on several datasets, still, evaluating the quality of conditionally generated samples is challenging. |
| Researcher Affiliation | Collaboration | 1AILAB, University of Trieste, Trieste, Italy 2Aindo, AREA Science Park, Trieste, Italy |
| Pseudocode | Yes | Algorithm 1: Constraint guidance sampling Input: constraint c(x), score s(x, t), score-based sampling algorithm A(s) Parameters: g(t), ϵ, n 1: sc(x, t) s(x, t) + g(t) xc(x) 2: x A( sc) 3: for i = 1 to n do 4: z N(0, 1) (with the dimensionality of x) 5: x x + ϵ sc(x, 0) + z 2ϵ 6: end for 7: return x |
| Open Source Code | Yes | Code https://github.com/Davide Scassola/score-basedconstrained-generation |
| Open Datasets | Yes | We made experiments with the white wine table of the popular UCI Wine Quality dataset (Paulo et al. 2009), consisting of 11 real-valued dimensions (R11) and one discrete dimension, the quality, that we discarded. In order to evaluate the effectiveness with categorical variables, we also made experiments with the Adult dataset (Becker and Kohavi 1996), consisting of 5 numerical dimensions and 10 categorical dimensions, which we embedded in a continuous space using one-hot encodings. |
| Dataset Splits | No | We fitted unconditional score-based diffusion models based on SDEs, then we generated samples and First we train a score-based generative model to fit trajectories that were generated by a simulator, with H = 30 and N = 100. This text does not specify how the datasets were split for training, validation, or testing. |
| Hardware Specification | No | This research was supported by Aindo, which has funded the Ph D of the first author and provided the computational resources. This statement is too general and does not provide specific hardware details. |
| Software Dependencies | No | The paper mentions using pre-trained unconditional models based on denoising score matching and SDEs, following closely Song et al. (2021). However, it does not provide specific software library names with version numbers. |
| Experiment Setup | Yes | When sampling, we can regulate the tradeoff between similarity with the original distribution and strength of the constraint by tuning the parameter k of inequalities in log-probabilistic logic. Alternatively, we can multiply by a constant λ the value of the constraint in order to scale its gradient. We experimented with various forms of g(t), mostly with the following two functions: Linear: g(t) = 1 t SNR: g(t) is equal to the signal-to-noise ratio of the diffusion kernel. For example, if the diffusion kernel qt( x|x) is N(x, σt), then g(t) = (1 + σ2 t ) 1 2 , assuming normalized data. In many of our experiments we found SNR to be the most effective, so we suggest using it as the first choice. [...] perform additional Langevin dynamics steps (these are referred to as corrector steps in Song et al. (2021)) at time t = 0 when the score of the constrained target distribution is known without approximation. Langevin dynamics can be used as a Monte Carlo method for sampling from a distribution when only the score is known (Welling and Teh 2011; Parisi 1981), performing the following update, where ϵ is the step size and zi is sampled from a standard normal with the dimensionality of x: xi+1 = xi + ϵ sc(x, 0) + zi 2ϵ |