Convergence of Consistency Model with Multistep Sampling under General Data Assumptions
Authors: Yiding Chen, Yiyi Zhang, Owen Oertell, Wen Sun
ICML 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Motivations: Consistency model has already demonstrated its power on large-scale image generation tasks (Luo et al., 2023; Song et al., 2023; Song & Dhariwal, 2024). To verify our theoretical findings, we focus on a toy example that is easier to interpret. We first refine our upper bound in Theorem 2, where we relax our result for a cleaner presentation. We make adjustment to (15) and get: supx,y supp(Pdata) x y 2 α2 t1 2σ2 t1 Ex Pdata h x 2 2 i + PN j=2 α2 tj 4σ2 tj t2 j 1 ϵ2 cm τ 2 1/4 + t N ϵcm τ . (26) Simulation setting: We consider OU process as the forward process, which is our setup in Case study 1. For simplicity, we consider a Bernoulli data distribution: Prx Pdata[x = 0] = Prx Pdata[x = 100] = 0.5. This data distribution ensures a close-form for the ground truth consistency function: f (x, t) := ( 0 if x < 50 exp( t) 100 o.w. . We construct a perturbed ˆf( , ) accordingly: ˆf(x, t) := ( 0 if x < at 100 o.w. , where the sequence at satisfies: Prx Pt[x < at] = 0.5 + 0.0001t2, t. This choice of ˆf( , ) makes sure: ˆf(x, t) f (x, t) 2 This means ˆf( , ) satisfies the first statement of Lemma 2 with ϵ2 cm τ 2 = 1. We simulate three instantiations of {ti}N i=1 defined in (5), i.e. the sequence of time steps for our multi-step sampling defined in (5): our schedule: the two-step schedule suggested by Case study 1. We also calculate the upper bound in (26) for comparison; baseline 1: design the sequence of sampling time steps by evenly dividing an interval; baseline 2: start with some T and reduce it by half every step until reaching a small value. In Figure 2, we plot the W2 error in multi-step sampling. We present the revolution of W2 error in a sampling time schedule on a single curve. Specifically, we plot each curve by: ti, W2( ˆP (i) 0 , Pdata) i = 1, ..., N. Because the sampling time step ti decreases in the multi-step sampling by definition. We reverse the x-axis of the plot for presentation purposes. Figure 2: W2 error in multi-step sampling. Observations: This simulation result demonstrates that: Our upper bound is a reasonable characterization of the performance for the designed sampling time schedule. The two-step sampling time schedule suggested by Case study 1 achieves comparable performance to the best result in the baseline methods but with a much smaller number of function evaluations; Running too many sampling time steps may degrade the sampling quality. The error increases for both baseline methods in the last few sampling steps. |
| Researcher Affiliation | Academia | 1Cornell University. Correspondence to: Yiding Chen <EMAIL>. |
| Pseudocode | Yes | For completeness, we summarize this process in Algorithm 1 in Section B. ... Algorithm 1 Multistep Consistency Sampling |
| Open Source Code | No | The paper does not provide any statement or link regarding the availability of source code for the methodology described. |
| Open Datasets | No | Simulation setting: We consider OU process as the forward process, which is our setup in Case study 1. For simplicity, we consider a Bernoulli data distribution: Prx Pdata[x = 0] = Prx Pdata[x = 100] = 0.5. This data distribution ensures a close-form for the ground truth consistency function: f (x, t) := ( 0 if x < 50 exp( t) 100 o.w. . We construct a perturbed ˆf( , ) accordingly: ˆf(x, t) := ( 0 if x < at 100 o.w. , where the sequence at satisfies: Prx Pt[x < at] = 0.5 + 0.0001t2, t. This choice of ˆf( , ) makes sure: ˆf(x, t) f (x, t) 2 This means ˆf( , ) satisfies the first statement of Lemma 2 with ϵ2 cm τ 2 = 1. The paper uses a custom-defined Bernoulli data distribution for its simulation and does not refer to any publicly available datasets with access information. |
| Dataset Splits | No | The paper uses a custom-defined Bernoulli data distribution for its simulation. It does not mention any training, validation, or test splits, as is typical for purely theoretical or toy example-based simulations. |
| Hardware Specification | No | The paper mentions "Our simulation in Appendix I" but does not provide any specific details about the hardware used to run this simulation or any other experiments. |
| Software Dependencies | No | The paper does not specify any software dependencies with version numbers. |
| Experiment Setup | Yes | Simulation setting: We consider OU process as the forward process, which is our setup in Case study 1. For simplicity, we consider a Bernoulli data distribution: Prx Pdata[x = 0] = Prx Pdata[x = 100] = 0.5. This data distribution ensures a close-form for the ground truth consistency function: f (x, t) := ( 0 if x < 50 exp( t) 100 o.w. . We construct a perturbed ˆf( , ) accordingly: ˆf(x, t) := ( 0 if x < at 100 o.w. , where the sequence at satisfies: Prx Pt[x < at] = 0.5 + 0.0001t2, t. This choice of ˆf( , ) makes sure: ˆf(x, t) f (x, t) 2 This means ˆf( , ) satisfies the first statement of Lemma 2 with ϵ2 cm τ 2 = 1. We simulate three instantiations of {ti}N i=1 defined in (5), i.e. the sequence of time steps for our multi-step sampling defined in (5): our schedule: the two-step schedule suggested by Case study 1. We also calculate the upper bound in (26) for comparison; baseline 1: design the sequence of sampling time steps by evenly dividing an interval; baseline 2: start with some T and reduce it by half every step until reaching a small value. |