arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.05367cs.CVcs.LG
收藏

LightCache:面向视频生成的内存高效、免训练加速方法

LightCache: Memory-Efficient, Training-Free Acceleration for Video Generation

University of Tulsa(塔尔萨大学);Clemson University(克莱姆斯大学);The University of Arizona(亚利桑那大学);Northeastern University(东北大学);Microsoft Research(微软研究院)

Yang Xiao, Gen Li, Kaiyuan Deng, Yushu Wu, Zheng Zhan, Yanzhi Wang, Xiaolong Ma, Bo Hui

AI总结:

针对视频生成中缓存加速导致的内存激增问题,提出分阶段策略(异步缓存交换、特征分块、切片解码),实现更快推理和更低内存,质量损失可控。

摘要:

免训练加速已成为基于扩散模型的视频生成领域的一个前沿研究方向。扩散模型推理过程中潜变量的冗余为加速提供了自然的切入点。在本文中,我们将推理过程分解为编码、去噪和解码三个阶段,并观察到基于缓存的加速方法往往会导致后两个阶段出现显著的内存激增。为解决这一问题,我们分析了不同阶段推理的特点,并提出了针对各阶段的内存消耗降低策略:1)异步缓存交换;2)特征分块;3)切片潜变量以进行解码。同时,我们确保这三种策略引入的时间开销低于其带来的加速收益。与基线方法相比,我们的方法实现了更快的推理速度和更低的内存占用,同时将质量下降控制在可接受的范围内。代码可在 https://github.com/NKUShaw/LightCache 获取。

英文摘要:

Training-free acceleration has emerged as an advanced research area in video generation based on diffusion models. The redundancy of latents in diffusion model inference provides a natural entry point for acceleration. In this paper, we decompose the inference process into the encoding, denoising, and decoding stages, and observe that cache-based acceleration methods often lead to substantial memory surges in the latter two stages. To address this problem, we analyze the characteristics of inference across different stages and propose stage-specific strategies for reducing memory consumption: 1) Asynchronous Cache Swapping. 2) Feature chunk. 3) Slicing latents to decode. At the same time, we ensure that the time overhead introduced by these three strategies remains lower than the acceleration gains themselves. Compared with the baseline, our approach achieves faster inference speed and lower memory usage, while maintaining quality degradation within an acceptable range. The Code is available at https://github.com/NKUShaw/LightCache .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑