arXiv 2505.241332026-01-22cs.CLcs.AI

R-KV：面向推理模型的冗余感知KV缓存压缩

R-KV: Redundancy-aware KV Cache Compression for Reasoning Models

University of Wisconsin - Madison（威斯康星大学麦迪逊分校）；Microsoft（微软）；Carnegie Mellon University（卡内基梅隆大学）；California Institute of Technology（加州理工学院）；University of California - San Diego（加州大学圣地亚哥分校）；University of Surrey（萨里大学）；University of California - Berkeley（加州大学伯克利分校）

Zefan Cai, Wen Xiao, Hanshi Sun, Cheng Luo, Yikai Zhang, Ke Wan, Yucheng Li, Yeyang Zhou, Li-Wen Chang, Jiuxiang Gu, Zhen Dong, Anima Anandkumar, Abedelkadir Asi, Junjie Hu

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

R-KV通过冗余感知机制实现推理模型KV缓存压缩，以10%的缓存占用率达到接近100%的性能，显著优于现有方法。

摘要：

推理模型在自我反思和链式推理任务中表现出色，但往往产生过长的输出，导致推理过程中键值（KV）缓存过大。尽管链式推理在复杂推理任务上显著提升了性能，但与现有KV缓存压缩方法结合时可能导致推理失败。为此，我们提出了针对推理模型冗余token的冗余感知KV缓存压缩（R-KV）方法。我们的方法仅使用10%的KV缓存即可保持接近100%的完整KV缓存性能，大幅优于现有KV缓存基线，后者仅能达到60%的性能。令人惊讶的是，R-KV甚至在使用16%的KV缓存时达到了105%的完整KV缓存性能。这种KV缓存减少还带来了90%的内存节省和标准链式推理推理的6.6倍吞吐量。实验结果表明，R-KV在两个数学推理数据集上均优于现有KV缓存压缩基线。

英文摘要：

Reasoning models have demonstrated impressive performance in self-reflection and chain-of-thought reasoning. However, they often produce excessively long outputs, leading to prohibitively large key-value (KV) caches during inference. While chain-of-thought inference significantly improves performance on complex reasoning tasks, it can also lead to reasoning failures when deployed with existing KV cache compression approaches. To address this, we propose Redundancy-aware KV Cache Compression for Reasoning models (R-KV), a novel method specifically targeting redundant tokens in reasoning models. Our method preserves nearly 100% of the full KV cache performance using only 10% of the KV cache, substantially outperforming existing KV cache baselines, which reach only 60% of the performance. Remarkably, R-KV even achieves 105% of full KV cache performance with 16% of the KV cache. This KV-cache reduction also leads to a 90% memory saving and a 6.6X throughput over standard chain-of-thought reasoning inference. Experimental results show that R-KV consistently outperforms existing KV cache compression baselines across two mathematical reasoning datasets.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。