Ca2-VDM: 具备因果生成与缓存共享的高效自回归视频扩散模型
Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharing
AI总结：
针对现有自回归视频扩散模型存在的冗余计算、复杂度随自回归步数二次增长的问题，提出具备因果生成与缓存共享的Ca2-VDM，消除冗余计算、降低缓存成本，实现生成质量与速度的双重提升。
摘要：
随着扩散模型的发展，如今的视频生成已取得令人瞩目的质量。为了延长生成时长并推动实际应用，大多数视频扩散模型（VDMs）采用自回归方式生成视频，即基于前一个片段的最后一帧或多帧生成后续片段。然而，现有自回归VDMs存在严重的低效与冗余问题：模型必须重新计算相邻片段之间重叠的所有条件帧。当为了给模型提供长期上下文而自回归扩展条件帧时，这一问题会进一步加剧。在这类场景下，计算需求会大幅上升，即计算复杂度与自回归步数呈二次方关系。本文提出Ca2-VDM，一种具备因果生成与缓存共享的高效自回归VDM。在因果生成方面，该模型引入单向特征计算，确保条件帧的缓存可在之前的自回归步骤中预先计算，并在后续每一步中复用，从而消除冗余计算。在缓存共享方面，它在所有去噪步骤之间共享缓存，避免了巨大的缓存存储成本。大量实验表明，我们的Ca2-VDM在视频生成的定量和定性结果上均达到了当前最优水平，并显著提升了生成速度。代码可在以下链接获取：https://github.com/Dawn-LX/CausalCache-VDM
英文摘要：
With the advance of diffusion models, today's video generation has achieved impressive quality. To extend the generation length and facilitate real-world applications, a majority of video diffusion models (VDMs) generate videos in an autoregressive manner, i.e., generating subsequent clips conditioned on the last frame(s) of the previous clip. However, existing autoregressive VDMs are highly inefficient and redundant: The model must re-compute all the conditional frames that are overlapped between adjacent clips. This issue is exacerbated when the conditional frames are extended autoregressively to provide the model with long-term context. In such cases, the computational demands increase significantly (i.e., with a quadratic complexity w.r.t. the autoregression step). In this paper, we propose Ca2-VDM, an efficient autoregressive VDM with Causal generation and Cache sharing. For causal generation, it introduces unidirectional feature computation, which ensures that the cache of conditional frames can be precomputed in previous autoregression steps and reused in every subsequent step, eliminating redundant computations. For cache sharing, it shares the cache across all denoising steps to avoid the huge cache storage cost. Extensive experiments demonstrated that our Ca2-VDM achieves state-of-the-art quantitative and qualitative video generation results and significantly improves the generation speed. Code is available: https://github.com/Dawn-LX/CausalCache-VDM
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
