VideoMLA: 用于分钟级自回归视频扩散的低秩潜在KV缓存
VideoMLA: Low-Rank Latent KV Cache for Minute-Scale Autoregressive Video Diffusion
AI总结：
本文提出VideoMLA，通过多头潜在注意力（MLA）将每头KV替换为共享低秩内容潜在和分离的3D-RoPE位置键，在视频扩散中减少92.7%的KV内存，并保持质量与吞吐量提升。
摘要：
长序列因果视频扩散已收敛于固定大小的滑动窗口KV缓存，近期创新通过改变窗口内令牌或位置编码方式在此布局内进行改进。每头KV布局本身是流式内存和延迟的主要贡献者，但基本保持不变。本文首次研究多头潜在注意力（MLA）在视频扩散中的应用。VideoMLA将每头的键和值替换为共享的低秩内容潜在和共享的解耦3D-RoPE位置键，在每个缓存层将每令牌KV内存减少92.7%。我们进一步探究了为什么MLA在视频扩散中成功，尽管语言模型中常用于激励它的谱假设不成立：预训练视频注意力不是低秩的，99%能量的有效秩远高于任何实际潜在维度。VideoMLA在压缩比下保持质量，而直接谱近似会预测较大的重构误差。我们表明，MLA瓶颈而非预训练谱决定了有效秩：谱和随机初始化都从初始化开始占据几乎全部秩预算，训练在此预算内适应。在VBench上，VideoMLA匹配短视界流式视频扩散基线，在长视界中取得最佳总体分数，并在单个B200上将吞吐量提升1.23倍。
英文摘要：
Long-rollout causal video diffusion has converged on a fixed-size sliding-window KV cache, with recent progress innovating within this layout by changing which tokens occupy the window or how their positions are encoded. The per-head KV layout itself, a dominant contributor to streaming memory and latency, has been mostly left unchanged. In this paper, we present the first study of Multi-Head Latent Attention (MLA) in video diffusion. VideoMLA replaces per-head keys and values with a shared low-rank content latent and a shared decoupled 3D-RoPE positional key, reducing per-token KV memory by 92.7% at every cached layer. We further investigate why MLA succeeds in video diffusion even though the spectral assumption often used to motivate it in language models does not hold: pretrained video attention is not low-rank, with 99%-energy effective rank far above any practical latent dimension. VideoMLA retains quality at compression ratios where direct spectral approximation would predict large reconstruction error. We show that the MLA bottleneck, rather than the pretrained spectrum, determines the effective rank: both spectral and random initialization occupy nearly the full rank budget from initialization, and training preserves this budget while adapting within it. On VBench, VideoMLA matches short-horizon streaming video diffusion baselines, achieves the best overall score at long horizons among evaluated methods, and improves throughput by 1.23x on a single B200.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
