arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.04304cs.CVcs.LG
收藏

Q-Diffusion:量化扩散模型

Q-Diffusion: Quantizing Diffusion Models

UC Berkeley(加州大学伯克利分校);Nanjing University(南京大学);UIUC(伊利诺伊大学厄巴纳-香槟分校);Peking University(北京大学)

Xiuyu Li, Yijiang Liu, Long Lian, Huanrui Yang, Zhen Dong, Daniel Kang, Shanghang Zhang, Kurt Keutzer

AI总结:

针对扩散模型噪声估计网络推理慢、传统PTQ量化效果差的问题,提出含时间步感知校准与分裂捷径量化的新型PTQ方法,首次实现4位权重的Stable Diffusion高质量生成。

摘要:

扩散模型通过利用深度神经网络进行迭代噪声估计,已在图像合成领域取得巨大成功。然而,噪声估计模型推理速度慢、内存消耗高且计算强度大,阻碍了扩散模型的高效应用。尽管训练后量化(PTQ)被视为其他任务的首选压缩方法,但它无法直接应用于扩散模型。我们提出了一种专门针对扩散模型独特的多时间步流程与模型架构设计的新型PTQ方法,该方法通过压缩噪声估计网络来加速生成过程。我们指出,扩散模型量化的核心难点在于噪声估计网络的输出分布会随多个时间步发生变化,且噪声估计网络内部的捷径层存在双峰激活分布。在本研究中,我们通过时间步感知校准与分裂捷径量化来解决这些挑战。实验结果表明,我们提出的方法能够在无需训练的情况下,将全精度无条件扩散模型量化为4位,同时保持相当的性能(FID变化最多仅为2.34,而传统PTQ的FID变化超过100)。我们的方法还可应用于文本引导的图像生成任务,首次实现了以4位权重运行Stable Diffusion并保持高生成质量。

英文摘要:

Diffusion models have achieved great success in image synthesis through iterative noise estimation using deep neural networks. However, the slow inference, high memory consumption, and computation intensity of the noise estimation model hinder the efficient adoption of diffusion models. Although post-training quantization (PTQ) is considered a go-to compression method for other tasks, it does not work out-of-the-box on diffusion models. We propose a novel PTQ method specifically tailored towards the unique multi-timestep pipeline and model architecture of the diffusion models, which compresses the noise estimation network to accelerate the generation process. We identify the key difficulty of diffusion model quantization as the changing output distributions of noise estimation networks over multiple time steps and the bimodal activation distribution of the shortcut layers within the noise estimation network. We tackle these challenges with timestep-aware calibration and split shortcut quantization in this work. Experimental results show that our proposed method is able to quantize full-precision unconditional diffusion models into 4-bit while maintaining comparable performance (small FID change of at most 2.34 compared to >100 for traditional PTQ) in a training-free manner. Our approach can also be applied to text-guided image generation, where we can run stable diffusion in 4-bit weights with high generation quality for the first time.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑