arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.03517cs.CVcs.AI
收藏

FRDiff:面向扩散模型通用免训练加速的特征复用

FRDiff : Feature Reuse for Universal Training-free Acceleration of Diffusion Models

POSTECH(浦项科技大学);Department of Computer Science and Engineering(计算机科学与工程系);Graduate School of Artificial Intelligence(人工智能研究生院)

Junhyuk So, Jungwon Lee, Eunhyeok Park

AI总结:

本文提出 FRDiff 方法,通过复用扩散模型中的高时间相似性特征并结合减少评分函数评估次数,在免训练条件下实现生成质量与计算延迟的帕累托最优加速。

摘要:

扩散模型高昂的计算成本,尤其是高质量图像生成所需的大量重复去噪步骤,是其广泛应用的主要障碍。尽管已有若干研究尝试通过使用先进的常微分方程求解器在不进行微调的情况下减少评分函数评估次数(NFE)来解决这一问题,但去噪迭代次数的减少会错失更新精细细节的机会,从而导致明显的质量下降。在本工作中,我们提出了一种利用扩散模型内在时间冗余的先进加速技术。复用具有高时间相似性的特征图为节省计算资源同时不损害输出质量开辟了新的可能。为了实现这一直觉的实际收益,我们进行了广泛的分析,并提出了一种新方法 FRDiff。FRDiff 旨在同时利用减少 NFE 和特征复用两方面的优势,在各类生成任务中实现兼顾保真度与延迟权衡的帕累托前沿。

英文摘要:

The substantial computational costs of diffusion models, especially due to the repeated denoising steps necessary for high-quality image generation, present a major obstacle to their widespread adoption. While several studies have attempted to address this issue by reducing the number of score function evaluations (NFE) using advanced ODE solvers without fine-tuning, the decreased number of denoising iterations misses the opportunity to update fine details, resulting in noticeable quality degradation. In our work, we introduce an advanced acceleration technique that leverages the temporal redundancy inherent in diffusion models. Reusing feature maps with high temporal similarity opens up a new opportunity to save computation resources without compromising output quality. To realize the practical benefits of this intuition, we conduct an extensive analysis and propose a novel method, FRDiff. FRDiff is designed to harness the advantages of both reduced NFE and feature reuse, achieving a Pareto frontier that balances fidelity and latency trade-offs in various generative tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑