ProphetKV: 基于用户查询的 selective recompute 方法用于高效 KV 缓存重用
ProphetKV: User-Query-Driven Selective Recomputation for Efficient KV Cache Reuse in Retrieval-Augmented Generation
AI总结:
ProphetKV通过用户查询驱动的KV缓存重用方法,高效恢复交叉注意力,提升检索增强生成的推理准确性。
摘要:
长上下文检索增强生成(RAG)的prefill阶段严重受限于计算开销。为缓解这一问题,近期方法通过用户查询检索RAG文档的预计算KV缓存,并重新处理选定的token以恢复这些预计算KV缓存之间的交叉注意力。然而,我们发现当前token选择标准中存在一个根本性的'crowding-out效应':全局显著但与用户查询无关的token会饱和有限的recompute预算,取代了真正回答用户查询所需的token,从而降低推理准确性。我们提出了ProphetKV,一种用于RAG场景的用户查询驱动的KV缓存重用方法。ProphetKV根据token与用户查询的语义相关性动态优先级,采用双阶段recompute流程将层间注意力指标融合为高实用性集合。通过确保recompute预算专门用于弥合检索上下文与用户查询之间的信息差距,ProphetKV在最小的开销下实现了高保真的注意力恢复。我们的广泛评估结果表明,ProphetKV在仅20%的recompute比率下保留了96%-101%的完整prefill准确性,在RULER上实现了8.8%-24.9%的准确率提升,在LongBench上实现了18.6%-50.9%的准确率提升,优于当前最先进的方法(如CacheBlend、EPIC和KVShare)
英文摘要:
The prefill stage of long-context Retrieval-Augmented Generation (RAG) is severely bottlenecked by computational overhead. To mitigate this, recent methods assemble pre-calculated KV caches of retrieved RAG documents (by a user query) and reprocess selected tokens to recover cross-attention between these pre-calculated KV caches. However, we identify a fundamental "crowding-out effect" in current token selection criteria: globally salient but user-query-irrelevant tokens saturate the limited recomputation budget, displacing the tokens truly essential for answering the user query and degrading inference accuracy. We propose ProphetKV, a user-query-driven KV Cache reuse method for RAG scenarios. ProphetKV dynamically prioritizes tokens based on their semantic relevance to the user query and employs a dual-stage recomputation pipeline to fuse layer-wise attention metrics into a high-utility set. By ensuring the recomputation budget is dedicated to bridging the informational gap between retrieved context and the user query, ProphetKV achieves high-fidelity attention recovery with minimal overhead. Our extensive evaluation results show that ProphetKV retains 96%-101% of full-prefill accuracy with only a 20% recomputation ratio, while achieving accuracy improvements of 8.8%-24.9% on RULER and 18.6%-50.9% on LongBench over the state-of-the-art approaches (e.g., CacheBlend, EPIC, and KVShare).
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。