轻量强制：通过稀疏注意力加速自回归视频扩散
Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attention
AI总结：
针对自回归视频生成模型中注意力二次复杂度问题，提出首个稀疏注意力方案Light Forcing，通过块感知增长机制和分层稀疏注意力实现质量和效率提升。
摘要：
先进的自回归（AR）视频生成模型提高了视觉保真度和交互性，但注意力的二次复杂度仍然是高效部署的主要瓶颈。虽然现有的稀疏注意力解决方案在双向模型上显示出潜力，但我们发现将这些解决方案应用于AR模型会导致显著的性能下降，原因有二：孤立地考虑块生成以及对过去信息上下文的利用不足。基于这些观察，我们提出 extsc{Light Forcing}，这是 extit{首个}专为AR视频生成模型设计的稀疏注意力解决方案。它引入了 extit{块感知增长}机制来定量估计每个块的贡献，从而决定其稀疏性分配。这种渐进式稀疏性增加策略使得当前块在生成过程中能够继承早期块中的先验知识。此外，我们引入了 extit{分层稀疏注意力}，以从粗到细的方式捕捉信息丰富的历史和局部上下文。这种两级掩码选择策略（即帧级和块级）能够自适应地处理不同的注意力模式。大量实验表明，我们的方法在质量（例如，VBench上84.5）和效率（例如，端到端加速1.2至1.3倍）上均优于现有的稀疏注意力方法。结合FP8量化和LightVAE， extsc{Light Forcing}在RTX 5090 GPU上进一步实现了2.3倍加速和19.7 FPS。代码将在\href{https://github.com/chengtao-lv/LightForcing}{https://github.com/chengtao-lv/LightForcing}发布。
英文摘要：
Advanced autoregressive (AR) video generation models have improved visual fidelity and interactivity, but the quadratic complexity of attention remains a primary bottleneck for efficient deployment. While existing sparse attention solutions have shown promise on bidirectional models, we identify that applying these solutions to AR models leads to considerable performance degradation for two reasons: isolated consideration of chunk generation and insufficient utilization of past informative context. Motivated by these observations, we propose \textsc{Light Forcing}, the \textit{first} sparse attention solution tailored for AR video generation models. It incorporates a \textit{Chunk-Aware Growth} mechanism to quantitatively estimate the contribution of each chunk, which determines their sparsity allocation. This progressive sparsity increase strategy enables the current chunk to inherit prior knowledge in earlier chunks during generation. Additionally, we introduce a \textit{Hierarchical Sparse Attention} to capture informative historical and local context in a coarse-to-fine manner. Such two-level mask selection strategy (i.e., frame and block level) can adaptively handle diverse attention patterns. Extensive experiments demonstrate that our method outperforms existing sparse attention in quality (e.g., 84.5 on VBench) and efficiency (e.g., $1.2{\sim}1.3\times$ end-to-end speedup). Combined with other efficient solutions, \textsc{Light Forcing} further achieves a $2.0{\sim}3.0\times$ end-to-end speedup across diverse GPUs (e.g., 27.4\,FPS on RTX 5090 and 33.9\,FPS on H100). Code is released via this \href{https://github.com/chengtao-lv/LightForcing}{link}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
