Large Language Models have excelled in various domains but face efficiency challenges due to the growing Key-Value (KV) cache required for long-sequence inference. Recent efforts aim to reduce KV cache size by evicting vast non-critical cache elements during runtime while preserving generation quality. However, these methods typically allocate compression budgets uniformly across all attention heads, ignoring the unique attention patterns of each head. In this paper, we establish a theoretical loss upper bound between pre- and post-eviction attention output, explaining the optimization target of prior cache eviction methods, while guiding the optimization of adaptive budget allocation. Base on this, we propose {\it Ada-KV}, the first head-wise adaptive budget allocation strategy. It offers plug-and-play benefits, enabling seamless integration with prior cache eviction methods. Extensive evaluations on 13 datasets from Ruler and 16 datasets from LongBench, all conducted under both question-aware and question-agnostic scenarios, demonstrate substantial quality improvements over existing methods. Our code is available at https://github.com/FFY0/AdaKV.
G-KV: Decoding-Time KV Cache Eviction with Global Attention
School of Computer Science and Technology, Beijing Jiaotong University（计算机科学与技术学院，北京交通大学）；MicroSoft（微软）；Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence（北京交通数据挖掘与具身智能重点实验室）
NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Time
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Baidu Inc.（百度公司）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
DeferKV: Rethinking Eviction Timing for One-Shot KV Cache Compression
University of Electronic Science and Technology of China（电子科技大学）；Ubiquitous Intelligence and Trusted Services Key Laboratory of Sichuan Province（四川省泛在智能与可信服务重点实验室）
SABlock: Semantic-Aware KV Cache Eviction with Adaptive Compression Block Size
School of Computer Science and Technology, University of Science and Technology of China（计算机科学与技术学院，科学技术大学）；Suzhou Institute for Advanced Research, University of Science and Technology of China（苏州先进研究院，科学技术大学）
Mask Tokens as Prophet: Fine-Grained Cache Eviction for Efficient dLLM Inference
School of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学人工智能学院）；Huazhong University of Science and Technology（华中科技大学）；University of Electronic Science and Technology of China（电子科技大学）；Carnegie Mellon University（卡内基梅隆大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
Taming the Fragility of KV Cache Eviction in LLM Inference
University of Science and Technology of China（中国科学技术大学）；USTC；Data Darkness Lab, MIRACLE Center, Suzhou Institute for Advanced Research（数据暗夜实验室、MIRACLE中心、苏州先进研究所）
Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective
College of Computer Science, Sichuan University, Chengdu, China（四川大学计算机学院，成都，中国）；Institute of High Performance Computing, A STAR, Singapore（高性能计算研究院，新加坡）
CriticalKV: Optimizing KV Cache Eviction from an Output Perturbation Perspective
School of Computer Science, University of Science and Technology of China（科学技术大学计算机科学学院）；School of Biomedical Engineering, USTC（USTC生物医学工程学院）；Data Darkness Lab, MIRACLE Center, Suzhou Institute for Advanced Research（苏州先进研究院数据黑暗实验室、奇迹中心）；School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）
ReST-KV: Robust KV Cache Eviction with Layer-wise Output Reconstruction and Spatial-Temporal Smoothing
Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences（基础模型研究中心，自动化研究所，中国科学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（人工智能学院，中国科学院大学）；University of Electronic Science and Technology of China（电子科技大学）；Wuhan AI Research（武汉人工智能研究）；Objecteye Inc.（Objecteye公司）
LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocation
State Key Laboratory for Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学）；School of Artificial Intelligence, Nanjing University（人工智能学院，南京大学）；Stepfun
LKV: End-to-End Learning of Head-wise Budgets and Token Selection for LLM KV Cache Eviction
University of Science and Technology of China（中国科学技术大学）；State Key Lab of Processors, Institute of Computing Technology, CAS, Beijing, China（中国科学院计算技术研究所过程器重点实验室，北京，中国）；University of Chinese Academy of Sciences, Beijing, China（中国科学院大学，北京，中国）
SpindleKV: A Novel KV Cache Reduction Method Balancing Both Shallow and Deep Layers
School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；School of Computer Science, Wuhan University（武汉大学计算机学院）；Xiaomi, Beijing, China（小米（北京，中国））；School of Information Management, Wuhan University（武汉大学信息管理学院）
