键、值、压缩：KV缓存压缩技术的系统探索
Key, Value, Compress: A Systematic Exploration of KV Cache Compression Techniques
AI总结：
本文系统探索了KV缓存压缩技术，提出分类法并评估其对性能和推理延迟的影响，揭示了压缩权衡，为高效处理长上下文的LLM实现提供指导。
摘要：
大型语言模型（LLMs）在生成文本、图像和视频内容方面展现了卓越的能力。然而，随着上下文长度的增长，注意力机制的计算成本随令牌数量呈二次方增加，带来了显著的效率挑战。本文对各种键值（KV）缓存压缩策略进行了分析，提供了一个全面的分类法，根据其基本原理和实现技术对这些方法进行分类。此外，我们评估了它们对性能和推理延迟的影响，提供了对其有效性的关键见解。我们的研究结果强调了KV缓存压缩中涉及的权衡及其对处理长上下文场景的影响，为更高效的LLM实现铺平了道路。
英文摘要：
Large language models (LLMs) have demonstrated exceptional capabilities in generating text, images, and video content. However, as context length grows, the computational cost of attention increases quadratically with the number of tokens, presenting significant efficiency challenges. This paper presents an analysis of various Key-Value (KV) cache compression strategies, offering a comprehensive taxonomy that categorizes these methods by their underlying principles and implementation techniques. Furthermore, we evaluate their impact on performance and inference latency, providing critical insights into their effectiveness. Our findings highlight the trade-offs involved in KV cache compression and its influence on handling long-context scenarios, paving the way for more efficient LLM implementations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
