The key-value (KV) cache is a foundational optimization in Transformer-based large language models (LLMs), eliminating redundant recomputation of past token representations during autoregressive generation. However, its memory footprint scales linearly with context length, imposing critical bottlenecks on GPU memory capacity, memory bandwidth, and inference throughput as production LLMs push context windows from thousands to millions of tokens. Efficient KV cache management has thus become a first-order challenge for scalable LLM deployment. This paper provides a systematic review of recent KV cache optimization techniques, organizing them into five principal directions: cache eviction, cache compression, hybrid memory solutions, novel attention mechanisms, and combination strategies. For each category we analyze the underlying mechanisms, deployment trade-offs, and empirical performance across memory reduction, throughput, and model accuracy metrics. We further map techniques to seven practical deployment scenarios, including long-context single requests, high-throughput datacenter serving, edge devices, multi-turn conversations, and accuracy-critical reasoning, providing actionable guidance for practitioners selecting among competing approaches. Our analysis reveals that no single technique dominates across all settings; instead, the optimal strategy depends on context length, hardware constraints, and workload characteristics, pointing toward adaptive, multi-stage optimization pipelines as a promising direction for future research.
A Survey on Large Language Model Acceleration based on KV Cache Management
Department of Computing, The Hong Kong Polytechnic University（计算系，香港理工大学）；Department of Computer Science and Engineering（计算机科学与工程系）；Department of Computer Science and Technology（计算机科学与技术系）；Department of Computing and Data Science（计算与数据科学系）
HybridKV: Hybrid KV Cache Compression for Efficient Multimodal Large Language Model Inference
The State Key Laboratory of Blockchain and Data Security, Zhejiang University（浙江大学区块链与数据安全国家重点实验室）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新区（滨江）区块链与数据安全研究院）；Hangzhou Dianzi University（杭州电子科技大学）
Integrative Sciences and Engineering Programme, National University of Singapore（国立新加坡大学整合科学与工程学程）；School of Computing, National University of Singapore（国立新加坡大学计算机学院）
Recency/Frequency Adaptive KV Caching for Large Language Model Serving
Department of Computer Science, Johns Hopkins University, Baltimore, USA（约翰霍普金斯大学计算机科学系）；Argonne National Laboratory, Lemont, USA（阿贡国家实验室）；Parasail, Inc., San Mateo, USA（Parasail公司）
SCALM: Towards Semantic Caching for Automated Chat Services with Large Language Models
School of Computing Science, Simon Fraser University（西蒙菲莎大学计算科学学院）；University of Mississippi（密西西比大学）；Jiangxing Intelligence Inc.（江兴智联公司）；The University of Hong Kong（香港大学）
Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving
The Hong Kong University of Science and Technology（香港科技大学）；Shanghai Jiao Tong University（上海交通大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
MTServe: Efficient Serving for Generative Recommendation Models with Hierarchical Caches
Institute for Clarity in Documentation（清晰文档研究所）；Inria Paris-Rocquencourt（巴黎-罗quant研究所）；Rajiv Gandhi University（拉吉夫·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕勒实验室）
EvolKV: Evolutionary KV Cache Compression for LLM Inference
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences（中国科学院大学先进交叉学科学院）；The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, CAS（复杂系统认知与决策智能重点实验室）；ETH Zurich（苏黎世联邦理工学院）
Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference
University of Connecticut Storrs（康涅狄格大学斯托尔斯分校）；Independent（独立研究者）；Indiana University Bloomington（印第安纳大学布卢明顿分校）；UC Santa Cruz（圣克拉拉大学）；Peking University（北京大学）
