GRKV: 长上下文LLM中免训练的KV缓存压缩的全局回归
GRKV: Global Regression for Training-Free KV Cache Compression in Long-Context LLMs
AI总结：
提出GRKV方法，通过岭回归合并步骤最小化压缩缓存与完整缓存注意力输出的差异，解决基于跨度保留的合并模式不平衡导致的过度合并和信息损失问题。
摘要：
具有扩展上下文长度的大型语言模型（LLM）依赖键值（KV）缓存来支持对先前令牌的注意力。然而，维护KV缓存会产生大量内存开销，促使通过驱逐和合并来强制执行固定预算的KV缓存压缩方法。现代驱逐方法越来越多地采用基于跨度的保留，因为保留连续跨度在经验上有效且更好地保持语义连贯性。然而，当与驱逐后合并结合时，基于跨度的保留将合并集中到一小部分跨度边界载体令牌上，产生高度不平衡的合并模式，加剧过度合并并增加信息损失。为了解决这种不平衡，我们提出GRKV（全局回归KV缓存），一种免训练的KV缓存合并方法，直接最小化压缩缓存与完整缓存注意力输出之间的差异。GRKV使用基于岭回归的合并步骤，将驱逐令牌的信息分布到保留令牌上，同时正则化更新以防止过度平滑。在LongBench和RULER长上下文基准测试中，GRKV是唯一一种以最小开销提高整体性能的合并方法。
英文摘要：
Large language models (LLMs) with extended context lengths rely on the key-value (KV) cache to support attention over prior tokens. However, maintaining the KV cache incurs substantial memory overhead, motivating KV-cache compression methods that enforce a fixed budget through eviction and merging. Modern eviction methods increasingly adopt span-based retention because preserving contiguous spans is empirically effective and better preserves semantic coherence. Yet, when combined with post-eviction merging, span-based retention concentrates merges onto a small set of span-boundary carrier tokens, producing a highly imbalanced merge pattern that exacerbates over-merging and increases information loss. To address this imbalance, we propose GRKV (Global Regression for KV Cache), a training-free KV-cache merging method that directly minimizes the discrepancy between compressed-cache and full-cache attention outputs. GRKV uses ridge-regression-based merge steps to distribute information from evicted tokens across retained tokens, while regularizing the updates to prevent over-smoothing. Across the LongBench and RULER long-context benchmarks, GRKV is the only merging method that improves overall performance with minimal overhead. Our code is available at https://github.com/pjunjie/GRKV.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
