超越快速一瞥：克服KV缓存压缩中的贪婪偏差
More Than a Quick Glance: Overcoming the Greedy Bias in KV-Cache Compression
AI总结：
LASER-KV通过块级累积策略克服KV缓存压缩中的贪婪偏差，实现更稳定的性能和更高的准确性。
摘要：
尽管大型语言模型（LLMs）理论上可以支持广泛的上下文窗口，但其实际部署却受到键值（KV）缓存内存线性增长的限制。现有的压缩策略通过各种剪枝机制来缓解这一问题，但往往以牺牲语义回忆为代价换取内存效率。在本工作中，我们提出了LASER-KV（层累积选择与精确LSH回忆）框架，旨在在严格的累积预算政策下测试KV压缩的极限。我们偏离标准的固定摘要大小方法，采用块级累积策略，由保护除数（n）控制。这使我们能够将压缩的影响与滑动窗口 artifacts 分离。我们在Babilong基准上的实验表明，先前的压缩方法在各种长上下文任务上性能下降了15-30%。LASER-KV保持了稳定的性能，在128k的情况下，通过10%的幅度实现了更高的准确性。这些发现挑战了普遍认为注意力分数本身足以作为token效用代理的假设。
英文摘要：
While Large Language Models (LLMs) can theoretically support extensive context windows, their actual deployment is constrained by the linear growth of Key-Value (KV) cache memory. Prevailing compression strategies mitigate this through various pruning mechanisms, yet trade-off semantic recall for memory efficiency. In this work, we present LASER-KV (Layer Accumulated Selection with Exact-LSH Recall), a framework designed to test the limits of KV compression under a strict accumulative budgeting policy. We deviate from the standard fixed summary size approach by implementing a block-wise accumulation strategy governed by a protection divisor (n). This allows us to isolate the effects of compression from sliding window artifacts. Our experiments on the Babilong benchmark reveal performance degradation in previous compression methods by 15-30% on various long context tasks. LASER-KV maintains stable performance, achieving superior accuracies by a margin of upto 10% at 128k. These findings challenge the prevailing assumption that attention scores alone are a sufficient proxy for token utility.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
