Recently, large language models (LLMs) have shown remarkable reasoning abilities by producing long reasoning traces. However, as the sequence length grows, the key-value (KV) cache expands linearly, incurring significant memory and computation costs. Existing KV cache eviction methods mitigate this issue by discarding less important KV pairs, but often fail to capture complex KV dependencies, resulting in performance degradation. To better balance efficiency and performance, we introduce ForesightKV, a training-based KV cache eviction framework that learns to predict which KV pairs to evict during long-text generations. We first design the Golden Eviction algorithm, which identifies the optimal eviction KV pairs at each step using future attention scores. These traces and the scores at each step are then distilled via supervised training with a Pairwise Ranking Loss. Furthermore, we formulate cache eviction as a Markov Decision Process and apply the GRPO algorithm to mitigate the significant language modeling loss increase on low-entropy tokens. Experiments on AIME2024 and AIME2025 benchmarks of three reasoning models demonstrate that ForesightKV consistently outperforms prior methods under only half the cache budget, while benefiting synergistically from both supervised and reinforcement learning approaches. Code is available at https://github.com/RUCAIBox/ForesightKV.
LKV: End-to-End Learning of Head-wise Budgets and Token Selection for LLM KV Cache Eviction
University of Science and Technology of China（中国科学技术大学）；State Key Lab of Processors, Institute of Computing Technology, CAS, Beijing, China（中国科学院计算技术研究所过程器重点实验室，北京，中国）；University of Chinese Academy of Sciences, Beijing, China（中国科学院大学，北京，中国）
NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Time
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Baidu Inc.（百度公司）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
DeferKV: Rethinking Eviction Timing for One-Shot KV Cache Compression
University of Electronic Science and Technology of China（电子科技大学）；Ubiquitous Intelligence and Trusted Services Key Laboratory of Sichuan Province（四川省泛在智能与可信服务重点实验室）
Forget Without Compromise: Nexus Sampling for Streaming KV-Cache Eviction Under Fixed Budgets
Department of Computer Science, Grinnell College（格林内尔学院计算机科学系）；Department of Computer Science, Rice University（莱斯大学计算机科学系）；Lambda, Inc（Lambda公司）；Workato（Workato公司）
G-KV: Decoding-Time KV Cache Eviction with Global Attention
School of Computer Science and Technology, Beijing Jiaotong University（计算机科学与技术学院，北京交通大学）；MicroSoft（微软）；Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence（北京交通数据挖掘与具身智能重点实验室）
SABlock: Semantic-Aware KV Cache Eviction with Adaptive Compression Block Size
School of Computer Science and Technology, University of Science and Technology of China（计算机科学与技术学院，科学技术大学）；Suzhou Institute for Advanced Research, University of Science and Technology of China（苏州先进研究院，科学技术大学）
Mask Tokens as Prophet: Fine-Grained Cache Eviction for Efficient dLLM Inference
School of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学人工智能学院）；Huazhong University of Science and Technology（华中科技大学）；University of Electronic Science and Technology of China（电子科技大学）；Carnegie Mellon University（卡内基梅隆大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
Taming the Fragility of KV Cache Eviction in LLM Inference
University of Science and Technology of China（中国科学技术大学）；USTC；Data Darkness Lab, MIRACLE Center, Suzhou Institute for Advanced Research（数据暗夜实验室、MIRACLE中心、苏州先进研究所）
Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective
College of Computer Science, Sichuan University, Chengdu, China（四川大学计算机学院，成都，中国）；Institute of High Performance Computing, A STAR, Singapore（高性能计算研究院，新加坡）
CriticalKV: Optimizing KV Cache Eviction from an Output Perturbation Perspective
School of Computer Science, University of Science and Technology of China（科学技术大学计算机科学学院）；School of Biomedical Engineering, USTC（USTC生物医学工程学院）；Data Darkness Lab, MIRACLE Center, Suzhou Institute for Advanced Research（苏州先进研究院数据黑暗实验室、奇迹中心）；School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）
ReST-KV: Robust KV Cache Eviction with Layer-wise Output Reconstruction and Spatial-Temporal Smoothing
Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences（基础模型研究中心，自动化研究所，中国科学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（人工智能学院，中国科学院大学）；University of Electronic Science and Technology of China（电子科技大学）；Wuhan AI Research（武汉人工智能研究）；Objecteye Inc.（Objecteye公司）
