强制头部：通过头部异质性实现长自回归视频生成
Head Forcing: Long Autoregressive Video Generation via Head Heterogeneity
AI总结：
本文提出Head Forcing框架，通过差异化分配KV缓存策略提升长视频生成质量，实现分钟级生成并支持多提示交互。
摘要：
自回归视频扩散模型支持实时合成但存在长时域误差累积和上下文丢失问题。我们发现AR视频扩散变换器中的注意力头部在功能上扮演着局部头部用于细节细化、锚头用于结构稳定化以及记忆头部用于长程上下文聚合的不同角色，但现有方法将其统一处理，导致KV缓存分配不优。我们提出Head Forcing，一种无需训练的框架，为每个头部类型分配定制化的KV缓存策略：局部和锚头仅保留必要token，而记忆头采用分层记忆系统，通过动态事件更新实现长程一致性。头部级RoPE重编码方案进一步确保位置编码保持在预训练范围内。无需额外训练，Head Forcing将生成时间从5秒扩展到分钟级，支持多提示交互合成，并一致优于现有基线。项目页面：https://jiahaotian-sjtu.github.io/headforcing.github.io/.
英文摘要：
Autoregressive video diffusion models support real-time synthesis but suffer from error accumulation and context loss over long horizons. We discover that attention heads in AR video diffusion transformers serve functionally distinct roles as local heads for detail refinement, anchor heads for structural stabilization, and memory heads for long-range context aggregation, yet existing methods treat them uniformly, leading to suboptimal KV cache allocation. We propose Head Forcing, a training-free framework that assigns each head type a tailored KV cache strategy: local and anchor heads retain only essential tokens, while memory heads employ a hierarchical memory system with dynamic episodic updates for long-range consistency. A head-wise RoPE re-encoding scheme further ensures positional encodings remain within the pretrained range. Without additional training, Head Forcing extends generation from 5 seconds to minute-level duration, supports multi-prompt interactive synthesis, and consistently outperforms existing baselines. Project Page: https://jiahaotian-sjtu.github.io/headforcing.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
