arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.19355cs.CV
收藏

FasterCache:无需训练的高质量视频扩散模型加速方法

FasterCache: Training-Free Video Diffusion Model Acceleration with High Quality

The University of Hong Kong(香港大学);Nanyang Technological University(南洋理工大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)

Zhengyao Lv, Chenyang Si, Junhao Song, Zhenyu Yang, Yu Qiao, Ziwei Liu, Kwan-Yee K. Wong

AI总结:

FasterCache提出无需训练的视频扩散模型加速方法,通过动态特征重用和CFG-Cache减少条件与无条件特征冗余,在保持视频质量的同时实现显著加速。

摘要:

本文提出了FasterCache,一种新颖的无需训练策略,旨在加速视频扩散模型的推理并保持高质量生成。通过分析现有基于缓存的方法,我们观察到直接重用相邻步特征会因丢失细微变化而降低视频质量。我们进一步对无分类器引导(CFG)的加速潜力进行了开创性研究,揭示了同一时间步内条件特征与无条件特征之间存在显著冗余。基于这些观察,我们引入FasterCache以大幅加速基于扩散的视频生成。我们的主要贡献包括:一种动态特征重用策略,能够同时保持特征区分度和时间连续性;以及CFG-Cache,它优化了条件输出与无条件输出的重用,从而在不牺牲视频质量的前提下进一步提升推理速度。我们在最近的视频扩散模型上对FasterCache进行了实证评估。实验结果表明,FasterCache能够显著加速视频生成(例如在Vchitect-2.0上实现1.67倍加速),同时保持与基线相当的视频质量,并在推理速度和视频质量上均一致优于现有方法。

英文摘要:

In this paper, we present \textbf{\textit{FasterCache}}, a novel training-free strategy designed to accelerate the inference of video diffusion models with high-quality generation. By analyzing existing cache-based methods, we observe that \textit{directly reusing adjacent-step features degrades video quality due to the loss of subtle variations}. We further perform a pioneering investigation of the acceleration potential of classifier-free guidance (CFG) and reveal significant redundancy between conditional and unconditional features within the same timestep. Capitalizing on these observations, we introduce FasterCache to substantially accelerate diffusion-based video generation. Our key contributions include a dynamic feature reuse strategy that preserves both feature distinction and temporal continuity, and CFG-Cache which optimizes the reuse of conditional and unconditional outputs to further enhance inference speed without compromising video quality. We empirically evaluate FasterCache on recent video diffusion models. Experimental results show that FasterCache can significantly accelerate video generation (\eg 1.67$\times$ speedup on Vchitect-2.0) while keeping video quality comparable to the baseline, and consistently outperform existing methods in both inference speed and video quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑