arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.14973cs.CLcs.AIcs.LG
收藏

注意力是扩散大语言模型中KV缓存的所有需求

Attention Is All You Need for KV Cache in Diffusion LLMs

MBZUAI(穆罕默德·本·扎耶德人工智能大学);FPT AI Residency(FPT人工智能驻留项目)

Quan Nguyen-Tri, Mukul Ranjan, Zhiqiang Shen

AI总结:

本文提出Elastic-Cache方法,通过适应性缓存更新提升扩散LLM的解码效率和生成质量。

摘要:

本工作研究如何适应性地重新计算扩散大语言模型(DLMs)中的键值(KV)缓存,以在最大化预测准确性的同时最小化解码延迟。先前方法的解码器在每个去噪步骤和层中都重新计算所有token的QKV,尽管大多数步骤中KV状态变化很小,特别是在浅层中,导致了显著的冗余。我们做出了三个观察:(1)远处的MASK token主要起长度偏倚作用,可以在活跃预测窗口之外按块缓存;(2)KV动态性随深度增加,表明从深层开始选择性刷新是足够的;(3)最被关注的token表现出最小的KV漂移,为其他token的缓存变化提供了保守的下限。基于这些,我们提出了Elastic-Cache,一种无需训练、架构无关的策略,共同决定何时刷新(通过针对最被关注的token的注意力感知漂移测试)和在哪里刷新(通过深度感知的计划,从选定的层开始重新计算,同时重用浅层缓存和非窗口MASK缓存)。与固定周期方案不同,Elastic-Cache为扩散LLM提供适应性、分层的缓存更新,减少冗余计算并加速解码,同时在生成质量上几乎没有损失。在LLaDA-Instruct、LLaDA-1.5和LLaDA-V上进行的实验表明,在数学推理和代码生成任务中实现了持续的速度提升:在GSM8K(256个token)上提升8.7倍,更长的序列上提升45.1倍,同时保持比基线更高的准确性。我们的方法在GSM8K上实现了显著更高的吞吐量(6.8倍),同时保持生成质量,使扩散LLM的实用部署成为可能。

英文摘要:

This work studies how to adaptively recompute key-value (KV) caches for diffusion large language models (DLMs) to maximize prediction accuracy while minimizing decoding latency. Prior methods' decoders recompute QKV for all tokens at every denoising step and layer, despite KV states changing little across most steps, especially in shallow layers, leading to substantial redundancy. We make three observations: (1) distant ${\bf MASK}$ tokens primarily act as a length-bias and can be cached block-wise beyond the active prediction window; (2) KV dynamics increase with depth, suggesting that selective refresh starting from deeper layers is sufficient; and (3) the most-attended token exhibits the smallest KV drift, providing a conservative lower bound on cache change for other tokens. Building on these, we propose ${\bf Elastic-Cache}$, a training-free, architecture-agnostic strategy that jointly decides ${when}$ to refresh (via an attention-aware drift test on the most-attended token) and ${where}$ to refresh (via a depth-aware schedule that recomputes from a chosen layer onward while reusing shallow-layer caches and off-window MASK caches). Unlike fixed-period schemes, Elastic-Cache performs adaptive, layer-aware cache updates for diffusion LLMs, reducing redundant computation and accelerating decoding with negligible loss in generation quality. Experiments on LLaDA-Instruct, LLaDA-1.5, and LLaDA-V across mathematical reasoning and code generation tasks demonstrate consistent speedups: $8.7\times$ on GSM8K (256 tokens), and $45.1\times$ on longer sequences, while consistently maintaining higher accuracy than the baseline. Our method achieves significantly higher throughput ($6.8\times$ on GSM8K) than existing confidence-based approaches while preserving generation quality, enabling practical deployment of diffusion LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑