arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.05595cs.CV
收藏

UniFL:通过统一反馈学习改进潜在扩散模型

UniFL: Improve Latent Diffusion Model via Unified Feedback Learning

Sun Yat-sen University(中山大学);Bytedance Inc(字节跳动公司);Peng Cheng Laboratory(鹏城实验室)

Jiacheng Zhang, Jie Wu, Yuxi Ren, Xin Xia, Huafeng Kuang, Pan Xie, Jiashi Li, Xuefeng Xiao, Weilin Huang, Shilei Wen, Lean Fu, Guanbin Li

AI总结:

针对当前潜在扩散模型(LDM)在视觉质量、美学吸引力和推理效率上的局限性,提出统一反馈学习框架UniFL,通过三个关键组件全面增强模型,在SD1.5、SDXL等模型上验证了优越性能。

摘要:

潜在扩散模型(LDM)彻底改变了文本到图像生成,推动了各种先进模型和多样化下游应用的普及。然而,尽管取得了这些显著进展,当前的扩散模型仍存在若干局限性,包括视觉质量较差、美学吸引力不足和推理效率低下,且尚无全面解决方案。为应对这些挑战,我们提出UniFL,一个利用反馈学习全面增强扩散模型的统一框架。UniFL是适用于SD1.5和SDXL等多种扩散模型的通用、有效且可泛化的解决方案。值得注意的是,UniFL包含三个关键组件:提升视觉质量的感知反馈学习、改善美学吸引力的解耦反馈学习,以及加速推理的对抗反馈学习。深入实验和广泛用户研究验证了我们的方法在增强生成质量和推理加速方面的优越性能。例如,在生成质量方面,UniFL在用户偏好上超过ImageReward 17%;在4步推理下,其总体偏好分别超过LCM 57%和SDXL Turbo 20%。

英文摘要:

Latent diffusion models (LDM) have revolutionized text-to-image generation, leading to the proliferation of various advanced models and diverse downstream applications. However, despite these significant advancements, current diffusion models still suffer from several limitations, including inferior visual quality, inadequate aesthetic appeal, and inefficient inference, without a comprehensive solution in sight. To address these challenges, we present UniFL, a unified framework that leverages feedback learning to enhance diffusion models comprehensively. UniFL stands out as a universal, effective, and generalizable solution applicable to various diffusion models, such as SD1.5 and SDXL. Notably, UniFL consists of three key components: perceptual feedback learning, which enhances visual quality; decoupled feedback learning, which improves aesthetic appeal; and adversarial feedback learning, which accelerates inference. In-depth experiments and extensive user studies validate the superior performance of our method in enhancing generation quality and inference acceleration. For instance, UniFL surpasses ImageReward by 17% user preference in terms of generation quality and outperforms LCM and SDXL Turbo by 57% and 20% general preference with 4-step inference.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑