xKV：通过对齐奇异向量提取的跨层KV缓存压缩
xKV: Cross-Layer KV-Cache Compression via Aligned Singular Vector Extraction
AI总结：
提出xKV，一种通过跨层共享低秩子空间压缩KV缓存的后训练方法，实现高达8倍压缩且保持长上下文任务精度，并引入选择性重建实现端到端加速。
摘要：
长上下文大型语言模型（LLMs）支持强大的应用，但由于键值状态（KV-Cache）导致高内存成本。最近的研究尝试跨层共享KV-Cache，但这些方法要么需要昂贵的预训练，要么依赖于实践中通常有限的逐token跨层余弦相似度。我们通过中心核对齐（CKA）表明，KV-Cache的主要奇异向量在层间对齐良好。受此观察启发，我们提出xKV，一种后训练压缩方法，将分组层的KV-Cache联合分解为共享的低秩子空间，大幅减少KV-Cache内存。在广泛使用的LLMs上，xKV实现了高达8倍的KV-Cache压缩，同时在长上下文任务和多轮设置中保持准确性。为进一步提高效率，我们在解码时引入选择性重建（SR）。结合SR，xKV相比全注意力基线实现了高达4.23倍的端到端加速，并在相似精度水平下以30%更高的吞吐量超越了显著基线。总体而言，xKV提供了一种即插即用的方法，用于减少长上下文LLM推理的内存和延迟。我们的代码公开于：https://github.com/abdelfattah-lab/xKV。
英文摘要：
Long-context Large Language Models (LLMs) enable powerful applications but incur high memory costs due to the key-value states (KV-Cache). Recent studies attempt to share KV-Cache across layers, but these approaches either require expensive pretraining or rely on per-token cross-layer cosine similarity that is often limited in practice. We show, via Centered Kernel Alignment (CKA), that the dominant singular vectors of KV-Cache are well aligned across layers. Motivated by this observation, we propose xKV, a post-training compression method that jointly factorizes grouped-layer KV-Cache into a shared low-rank subspace, substantially reducing KV-Cache memory. Across widely used LLMs, xKV achieves up to 8x KV-Cache compression while preserving accuracy on long-context tasks and in multi-turn settings. To further improve efficiency, we introduce Selective Reconstruction (SR) at decode time. Combined with SR, xKV achieves up to 4.23x end-to-end speedup over the full attention baseline, and surpasses notable baselines with 30% higher throughput under a similar accuracy level. Overall, xKV provides a plug-and-play approach to reduce both memory and latency for long-context LLM inference. Our code is publicly available at: https://github.com/abdelfattah-lab/xKV.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
