arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.05768cs.CVcs.AI
收藏

TLCM:用于2-8步图像生成的训练高效潜在一致性模型

TLCM: Training-efficient Latent Consistency Model for Image Generation with 2-8 Steps

AI Center, Guangdong OPPO Mobile Telecommunications Corp., Ltd(广东欧珀移动通信有限公司AI中心);Qing Yuan Research Institute, SEIEE, Shanghai Jiao Tong University(上海交通大学电子信息与电气工程学院清源研究院)

Qingsong Xie, Zhenyi Liao, Zhijie Deng, Chen chen, Haonan Lu

AI总结:

提出训练高效潜在一致性模型(TLCM),通过无数据多步潜在一致性蒸馏及多种技术,实现2-8步高质量图像生成,训练仅需70小时,性能超越教师模型。

摘要:

将潜在扩散模型(LDMs)蒸馏为采样快速的模型正引起越来越多的研究兴趣。然而,现有的大多数方法面临两个关键挑战:(1)它们依赖于使用大量真实数据进行长时间训练。(2)它们通常导致生成质量下降,尤其是在文本-图像对齐方面。本文提出了一种新颖的训练高效潜在一致性模型(TLCM)来克服这些挑战。我们的方法首先通过无数据多步潜在一致性蒸馏(MLCD)加速LDMs,然后提出无数据潜在一致性蒸馏以高效保证MLCD中的段间一致性。此外,我们引入了一系列技术,如分布匹配、对抗学习和偏好学习,以在无需任何真实数据的情况下增强TLCM在少步推理时的性能。TLCM通过允许在2到8范围内调整采样步骤,同时仍产生与全步方法相媲美的输出,展示了高度的灵活性。值得注意的是,TLCM利用教师模型生成的合成数据进行蒸馏,从而享有无数据的优势。仅需在A100 GPU上训练70小时,从SDXL蒸馏出的3步TLCM在MSCOCO-2017 5K基准上实现了令人印象深刻的CLIP分数33.68和美学分数5.97,超越了各种加速模型,甚至在人类偏好指标上超过了教师模型。我们还展示了TLCM在图像风格迁移、可控生成和中文到图像生成等应用中的多功能性。

英文摘要:

Distilling latent diffusion models (LDMs) into ones that are fast to sample from is attracting growing research interest. However, the majority of existing methods face two critical challenges: (1) They hinge on long training using a huge volume of real data. (2) They routinely lead to quality degradation for generation, especially in text-image alignment. This paper proposes a novel training-efficient Latent Consistency Model (TLCM) to overcome these challenges. Our method first accelerates LDMs via data-free multistep latent consistency distillation (MLCD), and then data-free latent consistency distillation is proposed to efficiently guarantee the inter-segment consistency in MLCD. Furthermore, we introduce bags of techniques, e.g., distribution matching, adversarial learning, and preference learning, to enhance TLCM's performance at few-step inference without any real data. TLCM demonstrates a high level of flexibility by enabling adjustment of sampling steps within the range of 2 to 8 while still producing competitive outputs compared to full-step approaches. Notably, TLCM enjoys the data-free merit by employing synthetic data from the teacher for distillation. With just 70 training hours on an A100 GPU, a 3-step TLCM distilled from SDXL achieves an impressive CLIP Score of 33.68 and an Aesthetic Score of 5.97 on the MSCOCO-2017 5K benchmark, surpassing various accelerated models and even outperforming the teacher model in human preference metrics. We also demonstrate the versatility of TLCMs in applications including image style transfer, controllable generation, and Chinese-to-image generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑