WnW：用于长文本语音大语言模型的消长型键值缓存
WnW: Waxing-and-Waning KV Cache for Long-Form Speech LLMs
浏览论文内容
中文总结 AI 辅助
针对长音频语音大语言模型的KV缓存内存开销问题，提出WnW消长型KV缓存方法，通过分类KV头实现高准确率与低GPU内存占用，且开销小、可迁移。
中文摘要 AI 辅助
长音频输入使键值（KV）缓存成为语音大语言模型的主要内存开销。仅预填充的KV压缩方法一旦驱逐音频KV位置就会永久丢弃，解码期间无法恢复。我们表明这在长音频上表现脆弱：预填充注意力集中在音频开头（注意力汇效应），而解码时注意力分布广泛，两者排名重叠度低。我们提出WnW（消长型KV缓存），通过离线校准将KV头分为锚定、潮汐和固定角色：锚定头保留在GPU上，作为解码时的重要性观测器；潮汐头保留CPU驻留的补充部分，基于聚合锚定头得分按块召回；固定头仅保留GPU子集，其余永久丢弃。在LibriSpeech-Long数据集上，使用两个3B参数主干模型Voxtral-mini-3b和Qwen2.5-Omni-3B，WnW在仅保留20%音频标记在GPU上的同时，保持接近全缓存的准确率，而仅预填充的基线方法无法完成任务。结果可跨语言、任务和领域迁移，且CPU-GPU召回在我们的测量中几乎不增加解码开销。
英文摘要
Long-form audio inputs make the KV cache the dominant memory cost of speech LLMs. Prefill-only KV compression methods permanently discard audio KV positions once evicted, with no pathway to recover them during decoding. We show this is fragile on long-form audio: prefill attention concentrates near the audio start (an attention-sink effect), while decode-time attention distributes broadly, and the two rankings overlap weakly. We propose WnW (Waxing-and-Waning KV cache), which classifies KV-heads into anchor, tidal, and fixed roles via offline calibration. Anchor heads keep all audio KV on GPU and yield a decode-time signal of which audio region each token is read from; tidal heads keep a CPU-resident complement that is recalled chunk-by-chunk based on aggregated anchor-head scores; fixed heads keep only an on-GPU subset, with the rest permanently discarded. On LibriSpeech-Long with two 3B backbones (Voxtral-mini-3b and Qwen2.5-Omni-3B), WnW preserves near-Full-Cache accuracy while keeping only 20% of audio tokens on GPU, where prefill-only baselines fail to terminate. Results generalize across language, task, and domain shifts, and CPU-GPU recall adds little decode-time overhead in our measurements.
发表机构
- Alibaba Group（阿里巴巴集团）
- School of Informatics Xiamen University（厦门大学信息学院）
机构由 AI 辅助整理，请以论文原文为准。
