稀疏强制：用于实时自回归扩散视频生成的原生可训练稀疏注意力
Sparse Forcing: Native Trainable Sparse Attention for Real-time Autoregressive Diffusion Video Generation
AI总结：
本文提出稀疏强制方法，通过在训练和推理中提升长时程生成质量并降低解码延迟，采用可训练的原生稀疏机制和高效的GPU内核PBSA来加速稀疏注意力和内存更新。
摘要：
我们介绍了稀疏强制，一种用于自回归视频扩散模型的训练和推理范式，旨在在提高长时程生成质量的同时减少解码延迟。稀疏强制受到自回归扩散滚动中的经验观察的启发：注意力集中在持久的显著视觉块上，形成隐含的时空记忆在KV缓存中，并在滑动窗口内表现出局部结构化的块稀疏模式。基于这一观察，我们提出了一种可训练的原生稀疏机制，该机制学习压缩、保留和更新这些持久块，同时在每个局部窗口内限制计算到动态选择的局部邻域。为了使该方法在大规模训练和推理中实用，我们进一步提出持久块稀疏注意力（PBSA），一种高效的GPU内核，可加速稀疏注意力和内存更新，以实现低延迟、内存高效的解码。实验表明，稀疏强制在5秒文本到视频生成中比自强制提高了+0.26的VBench分数，同时提供1.11-1.17倍的解码加速和42%更低的峰值KV缓存足迹。在更长的时程滚动中，收益更加显著，提供改进的视觉质量+0.68和+2.74的VBench改进，并在20秒和1分钟生成中分别提供1.22倍和1.27倍的加速。
英文摘要：
We introduce Sparse Forcing, a training-and-inference paradigm for autoregressive video diffusion models that improves long-horizon generation quality while reducing decoding latency. Sparse Forcing is motivated by an empirical observation in autoregressive diffusion rollouts: attention concentrates on a persistent subset of salient visual blocks, forming an implicit spatiotemporal memory in the KV cache, and exhibits a locally structured block-sparse pattern within sliding windows. Building on this observation, we propose a trainable native sparsity mechanism that learns to compress, preserve, and update these persistent blocks while restricting computation within each local window to a dynamically selected local neighborhood. To make the approach practical at scale for both training and inference, we further propose Persistent Block-Sparse Attention (PBSA), an efficient GPU kernel that accelerates sparse attention and memory updates for low-latency, memory-efficient decoding. Experiments show that Sparse Forcing improves the VBench score by +0.26 over Self-Forcing on 5-second text-to-video generation while delivering a 1.11-1.17x decoding speedup and 42% lower peak KV-cache footprint. The gains are more pronounced on longer-horizon rollouts, delivering improved visual quality with +0.68 and +2.74 VBench improvements, and 1.22x and 1.27x speedups on 20-second and 1-minute generations, respectively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
