一个尺寸并不适合所有：针对KV缓存的令牌级自适应压缩
One Size Does Not Fit All: Token-Wise Adaptive Compression for KV Cache
AI总结：
DynaKV 是一种基于令牌级自适应压缩的KV缓存压缩方法，通过动态分配压缩率提升压缩比下的保真度，实现内存减少与生成质量的平衡。
摘要：
尽管大型语言模型（LLMs）取得了显著进展，但Key-Value（KV）缓存的快速增长内存足迹仍然是高效推理的关键瓶颈。虽然降维缩减提供了一种有前途的压缩途径，但现有方法通常要么需要 prohibitively expensive 的从头预训练，要么在高压缩条件下导致严重的性能下降。在本工作中，我们提出 DynaKV，一种新颖的后训练框架用于低秩KV缓存压缩。据我们所知，DynaKV 是第一个根据语义意义动态分配压缩率到单个令牌的方法，这使它能够在高压缩比下实现更好的保真度。广泛的实验表明，我们的方法在现有最先进压缩技术中表现一致，实现了显著的内存减少，同时保持竞争性的生成质量。此外，我们的方法与序列级剪枝方法是正交的。当与 SnapKV 集成时，DynaKV 在 LongBench 检验中仅保留 6% 的 KV 缓存，同时保持 94% 的基线性能。
英文摘要：
Despite the remarkable progress of Large Language Models (LLMs), the escalating memory footprint of the Key-Value (KV) cache remains a critical bottleneck for efficient inference. While dimensionality reduction offers a promising compression avenue, existing approaches typically either necessitate prohibitively expensive pre-training from scratch or suffer from severe performance deterioration under high compression regimes. In this work, we propose DynaKV, a novel post-training framework for low-rank KV cache compression. To the best of our knowledge, DynaKV is the first method to dynamically allocate compression rates to individual tokens according to their semantic meaning, which allows it to achieve better fidelity at aggressive compression ratios. Extensive experiments demonstrate that our method consistently outperforms existing state-of-the-art compression techniques, achieving significant memory reduction while maintaining competitive generation quality. Furthermore, our approach is orthogonal to sequence-level pruning methods. When integrated with SnapKV, DynaKV retains only 6% of the KV cache while maintaining 94% of the baseline performance on the LongBench benchmark.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
