Diffusion models are relatively easy to train but require many steps to generate samples. Consistency models are far more difficult to train, but generate samples in a single step.
In this paper we propose Multistep Consistency Models: A unification between Consistency Models (Song et al., 2023) and TRACT (Berthelot et al., 2023) that can interpolate between a consistency model and a diffusion model: a trade-off between sampling speed and sampling quality. Specifically, a 1-step consistency model is a conventional consistency model whereas a $\infty$-step consistency model is a diffusion model.
Multistep Consistency Models work really well in practice. By increasing the sample budget from a single step to 2-8 steps, we can train models more easily that generate higher quality samples, while retaining much of the sampling speed benefits. Notable results are 1.4 FID on Imagenet 64 in 8 step and 2.1 FID on Imagenet128 in 8 steps with consistency distillation, using simple losses without adversarial training. We also show that our method scales to a text-to-image diffusion model, generating samples that are close to the quality of the original model.
Predict to Skip: Linear Multistep Feature Forecasting for Efficient Diffusion Transformers
School of Artificial Intelligence, Beijing Normal University, Beijing 100875, China(人工智能学院,北京师范大学,北京);Institute of Artificial Intelligence(人工智能研究所);Future Networks, Beijing Normal University, Zhuhai 519087, China(未来网络,北京师范大学,珠海)
TLCM: Training-efficient Latent Consistency Model for Image Generation with 2-8 Steps
AI Center, Guangdong OPPO Mobile Telecommunications Corp., Ltd(广东欧珀移动通信有限公司AI中心);Qing Yuan Research Institute, SEIEE, Shanghai Jiao Tong University(上海交通大学电子信息与电气工程学院清源研究院)
Stabilizing Consistency Training: A Flow Map Analysis and Self-Distillation
Department of Computer Science and Engineering(计算机科学与工程系);Seoul National University(首尔国立大学);Interdisciplinary Program in Artificial Intelligence(人工智能跨学科项目)
ACT-Diffusion: Efficient Adversarial Consistency Training for One-step Diffusion Models
University of Electronic Science and Technology of China(电子科技大学);Drexel University(德雷塞尔大学);Lehigh University(理海大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Institute of Artificial Intelligence Innovation, National Yang Ming Chiao Tung University(国立阳明交通大学人工智慧创新学院);National Center for Theoretical Sciences, National Taiwan University(台湾大学国家理论科学中心)
Enhancing Low Dose Computed Tomography Images Using Consistency Training Techniques
University of Kentucky(肯塔基大学);Institute for Biomedical Informatics(生物医学信息学研究所);Rensselaer Polytechnic Institute(伦斯勒理工学院);University of Alabama at Birmingham(阿拉巴马大学伯明翰分校)
Pratt School of Engineering Duke University(普拉特工程学院 哥伦比亚大学);Biomedical Imaging Center Rensselaer Polytechnic Institute(生物医学成像中心 莱文森理工学院);Department of Computer Science University of Kentucky(计算机科学系 肯塔基大学);Department of Medicine Department of Biomedical Informations University of Alabama at Birmingham(医学系 生物医学信息学系 亚拉巴马大学伯明翰分校);School of Engineering Sciences Department of Physics KTH Royal Institute of Technology(工程科学学院 物理系 瑞典皇家理工学院)
Iterating Consistency Models: Stability, Error Bounds and Noise Schedules
Université Paris Cité(巴黎西岱大学);CNRS(法国国家科学研究中心);Heriot-Watt University(赫瑞-瓦特大学);École Polytechnique(巴黎综合理工学院);EPITA(法国计算机科学与技术高等学院);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Stabilizing Physics-Informed Consistency Models via Structure-Preserving Training
Institute of Artificial Intelligence Innovation(人工智能创新研究所);National Yang Ming Chiao Tung University(国立阳明交通大学);Department of Applied Mathematics(应用数学系);National Center for Theoretical Sciences(理论科学研究中心);National Taiwan University(国立台湾大学)