语义完整性至关重要：在KV缓存压缩中基准测试与保持高密度推理
Semantic Integrity Matters: Benchmarking and Preserving High-Density Reasoning in KV Cache Compression
AI总结：
本文提出KVFundaBench基准测试，揭示了在高密度推理中压缩导致的严重任务依赖性退化问题，并提出ShotKV方法，通过分离prefill和解码阶段以优先保持语义完整性，提升了长上下文生成任务的准确率并降低了延迟。
摘要：
尽管键值（KV）缓存压缩对高效LLM推理至关重要，但当前评估过度关注稀疏检索任务，可能掩盖了高密度推理中链式思维（CoT）连贯性至关重要的退化问题。我们引入KVFundaBench系统评估这一差距，揭示出一个鲜明的二元对立：检索任务仍保持稳健，但推理任务在激进压缩下因CoT链接中断而表现出严重的任务依赖性退化。扩展分析至DeepSeek-R1模型，我们发现其专用注意力模式提供了关于推理链脆弱性的独特见解。基于这些发现——特别是保持少量示例作为不可分割的语义单元的必要性——我们提出ShotKV。该方法明确分离prefill和解码阶段以优先考虑语义完整性。实验结果表明，ShotKV在长上下文生成任务中实现了9%-18%的准确率提升，并有效扩展到文档问答，同时相比完整缓存推理实现了11%的延迟降低。
英文摘要：
While Key-Value (KV) cache compression is essential for efficient LLM inference, current evaluations disproportionately focus on sparse retrieval tasks, potentially masking the degradation of High-Density Reasoning where Chain-of-Thought (CoT) coherence is critical. We introduce KVFundaBench to systematically evaluate this gap, revealing a sharp dichotomy: while retrieval tasks remain robust, reasoning tasks exhibit severe Task-Dependent Degradation under aggressive compression due to disrupted CoT links. Extending our analysis to the DeepSeek-R1 model, we uncover that its specialized attention patterns offer unique insights into the fragility of reasoning chains. Guided by these findings -- specifically the necessity of preserving few-shot examples as indivisible Semantic Units -- we propose ShotKV. This approach explicitly separates prefill and decoding phases to prioritize semantic integrity. Empirical results demonstrate that ShotKV achieves 9%-18% accuracy improvements on long-context generation tasks and effectively generalizes to document QA, all while delivering an 11% latency reduction compared to full cache inference.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
