Consistency models, which were proposed to mitigate the high computational overhead during the sampling phase of diffusion models, facilitate single-step sampling while attaining state-of-the-art empirical performance. When integrated into the training phase, consistency models attempt to train a sequence of consistency functions capable of mapping any point at any time step of the diffusion process to its starting point. Despite the empirical success, a comprehensive theoretical understanding of consistency training remains elusive. This paper takes a first step towards establishing theoretical underpinnings for consistency models. We demonstrate that, in order to generate samples within $\varepsilon$ proximity to the target in distribution (measured by some Wasserstein metric), it suffices for the number of steps in consistency learning to exceed the order of $d^{5/2}/\varepsilon$, with $d$ the data dimension. Our theory offers rigorous insights into the validity and efficacy of consistency models, illuminating their utility in downstream inference tasks.
On the Memorization of Consistency Distillation for Diffusion Models
School of Computing & Data Science, The University of Hong Kong（计算与数据科学学院，香港大学）；School of Computing & Data Science and Institute of Data Science, The University of Hong Kong（计算与数据科学学院和数据科学研究所，香港大学）
Decoupled Data Consistency with Diffusion Purification for Image Restoration
Department of Electrical Engineering and Computer Science, University of Michigan（电气工程与计算机科学系，密歇根大学）；Departments of Computational Mathematics, Science and Engineering, and Biomedical Engineering, Michigan State University（计算数学、科学与工程系，以及生物医学工程系，密歇根州立大学）
