金字塔强制：面向高质量长视频生成的头部感知金字塔KV缓存策略
Pyramid Forcing: Head-Aware Pyramid KV Cache Policy for High-Quality Long Video Generation
AI总结：
本文提出Pyramid Forcing，通过识别不同头部类型并分配特定缓存策略，提升长视频生成质量，实验显示在VBench-Long上Self Forcing得分提升。
摘要：
自回归视频生成能够实现流式和开放式的长视频合成，但仍然受到累积误差导致的长期退化影响。现有KVCache策略通常采用统一的历史帧保留方法，隐含假设所有注意力头的历史依赖是同质的。我们重新审视历史帧注意力，揭示三种不同的头部类型：锚头需要广泛的长距离上下文，波形头表现出周期性的时间依赖性，遮蔽头专注于初始和相邻帧。基于此发现，我们提出Pyramid Forcing，一种头部感知的金字塔KV缓存框架，该框架在离线阶段识别头部类型，分配行为特定的缓存策略，并通过高效的不规则缓存注意力支持异质缓存长度。在Self Forcing和Causal Forcing上的实验表明，Pyramid Forcing在VBench-Long上一致提升了长视图生成质量，将60秒Self Forcing得分从77.87提高到81.21，同时增强了运动动态性、视觉保真度和语义一致性。项目：https://if-lab-pku.github.io/Pyramid-Forcing/.
英文摘要：
Autoregressive video generation enables streaming and open-ended long video synthesis, but still suffers from long-term degradation caused by accumulated errors. Existing KVCache strategies usually apply unified historical-frame retention, implicitly assuming homogeneous historical dependencies across attention heads. We revisit historical-frame attention and reveal three distinct head types: Anchor Heads require broad long-range context, Wave Heads exhibit periodic temporal dependencies, and Veil Heads focus on initial and adjacent frames. Based on this finding, we propose Pyramid Forcing, a head-aware pyramidal KVCache framework that identifies head types offline, assigns behavior-specific cache policies, and supports heterogeneous cache lengths via efficient ragged-cache attention. Experiments on Self Forcing and Causal Forcing show that Pyramid Forcing consistently improves long-horizon generation quality on VBench-Long, increasing the 60-second Self Forcing score from 77.87 to 81.21 while enhancing motion dynamics, visual fidelity, and semantic consistency. Project: https://if-lab-pku.github.io/Pyramid-Forcing/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
