聚焦强制：面向内容的每帧KV选择用于高效的自回归视频扩散
Focused Forcing: Content-Aware Per-Frame KV Selection for Efficient Autoregressive Video Diffusion
AI总结：
本文提出了一种无需训练的KV选择方法，通过结合注意力分数和历史帧的多样性分数，保留最相关和有区别的历史帧，从而在不牺牲质量的情况下提高自回归视频扩散的效率。
摘要：
近期在自回归视频扩散领域的进展使得序列和流式视频生成成为可能。然而，长视界生成需要越来越大的KV缓存，这使得在不牺牲质量的情况下实现高效的压缩具有挑战性。现有方法大多基于注意力分数选择历史帧，但它们的上下文决策仍然粗略。当同一块中生成多个帧时，这些方法通常对整个块应用共享的历史选择，仅通过注意力对历史帧评分，并将头预算均匀或通过注意力模式启发式分配，而不是显式估计头重要性。我们发现同一生成块中的帧可能依赖于不同的历史帧，同一历史帧在与当前帧的相对时间距离变化时可能获得不同的注意力分数，且屏蔽不同头会引发不均等的生成退化。受这些发现的启发，我们提出了Focused Forcing，一种无需训练的KV选择方法，该方法在生成帧和头维度上聚焦缓存历史。对于每个生成帧，Focused Forcing通过结合注意力分数和历史帧的多样性分数保留最相关和有区别的历史帧，同时将较大的预算分配给估计重要性更高的头。在多个自回归生成范式中，Focused Forcing在不训练的情况下实现了高达1.48倍的端到端加速，同时提高了视觉质量和文本对齐。
英文摘要：
Recent advances in autoregressive video diffusion have enabled sequential and streaming video generation. However, long-horizon generation requires increasingly large KV caches, making efficient compression without sacrificing quality challenging. Existing methods mostly select historical frames based on attention scores, but their context decisions remain coarse. When multiple frames are generated in the same chunk, these methods often apply a shared history selection to the whole chunk, score historical frames solely by attention, and assign head-wise budgets either uniformly or by attention-pattern heuristics rather than explicit head-importance estimation. We show that frames within the same generated chunk can depend on distinct historical frames, that the same historical frame can receive different attention scores as its relative temporal distance to the current frames changes, and that masking different heads induces unequal generation degradation. Motivated by these findings, we propose \textbf{Focused Forcing}, a training-free KV selection method that focuses cached history along both generated-frame and head dimensions. For each generated frame, Focused Forcing preserves the most relevant and distinctive historical frames by combining attention scores with diversity scores of historical frames, while assigning larger budgets to heads with higher estimated importance. Across multiple autoregressive generation paradigms, Focused Forcing achieves up to $\textbf{1.48}\times$ end-to-end acceleration without training, while \textbf{improving visual quality and text alignment}. \textit{Our code will be released on GitHub.}
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
