arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.19140cs.CVcs.AI
收藏

QNCD:面向扩散模型的量化噪声校正

QNCD: Quantization Noise Correction for Diffusion Models

Kuaishou Technology(快手科技);Meituan(美团)

Huanpeng Chu, Wei Wu, Chengjie Zang, Kun Yuan

AI总结:

针对扩散模型训练后量化在低比特下样本质量下降的问题,提出QNCD方案,通过特征平滑和运行时噪声估计分别消除层内、层间量化噪声,在ImageNet的W4A8、W8A8量化下实现无损效果。

摘要:

扩散模型已彻底革新图像合成领域,在质量与创造性上树立了新基准,但其迭代去噪过程所需的密集计算阻碍了广泛应用。训练后量化(PTQ)为加速采样提供了解决方案,但会牺牲样本质量,在低比特设置下尤为严重。针对这一问题,本研究提出统一的量化噪声校正方案QNCD,旨在降低整个采样过程中的量化噪声。我们识别出两类核心量化挑战:层内量化噪声与层间量化噪声。层内量化噪声主要由残差块(resblock)模块中的嵌入向量加剧,会扩大激活量化范围,增加单步去噪过程中的扰动。此外,层间量化噪声源于整个去噪过程中累积的量化偏差,会逐步改变数据分布。QNCD通过基于嵌入的特征平滑消除层内量化噪声,并通过高效的运行时噪声估计模块动态滤除层间量化噪声,以此应对上述问题。大量实验表明,本方法性能优于已有的扩散模型量化方法,在ImageNet数据集(LDM-4)的W4A8和W8A8量化设置下实现了无损效果。代码可在https://github.com/huanpengchu/QNCD获取。

英文摘要:

Diffusion models have revolutionized image synthesis, setting new benchmarks in quality and creativity. However, their widespread adoption is hindered by the intensive computation required during the iterative denoising process. Post-training quantization (PTQ) presents a solution to accelerate sampling, aibeit at the expense of sample quality, extremely in low-bit settings. Addressing this, our study introduces a unified Quantization Noise Correction Scheme (QNCD), aimed at minishing quantization noise throughout the sampling process. We identify two primary quantization challenges: intra and inter quantization noise. Intra quantization noise, mainly exacerbated by embeddings in the resblock module, extends activation quantization ranges, increasing disturbances in each single denosing step. Besides, inter quantization noise stems from cumulative quantization deviations across the entire denoising process, altering data distributions step-by-step. QNCD combats these through embedding-derived feature smoothing for eliminating intra quantization noise and an effective runtime noise estimatiation module for dynamicly filtering inter quantization noise. Extensive experiments demonstrate that our method outperforms previous quantization methods for diffusion models, achieving lossless results in W4A8 and W8A8 quantization settings on ImageNet (LDM-4). Code is available at: https://github.com/huanpengchu/QNCD

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑