This work studies how to adaptively recompute key-value (KV) caches for diffusion large language models (DLMs) to maximize prediction accuracy while minimizing decoding latency. Prior methods' decoders recompute QKV for all tokens at every denoising step and layer, despite KV states changing little across most steps, especially in shallow layers, leading to substantial redundancy. We make three observations: (1) distant ${\bf MASK}$ tokens primarily act as a length-bias and can be cached block-wise beyond the active prediction window; (2) KV dynamics increase with depth, suggesting that selective refresh starting from deeper layers is sufficient; and (3) the most-attended token exhibits the smallest KV drift, providing a conservative lower bound on cache change for other tokens. Building on these, we propose ${\bf Elastic-Cache}$, a training-free, architecture-agnostic strategy that jointly decides ${when}$ to refresh (via an attention-aware drift test on the most-attended token) and ${where}$ to refresh (via a depth-aware schedule that recomputes from a chosen layer onward while reusing shallow-layer caches and off-window MASK caches). Unlike fixed-period schemes, Elastic-Cache performs adaptive, layer-aware cache updates for diffusion LLMs, reducing redundant computation and accelerating decoding with negligible loss in generation quality. Experiments on LLaDA-Instruct, LLaDA-1.5, and LLaDA-V across mathematical reasoning and code generation tasks demonstrate consistent speedups: $8.7\times$ on GSM8K (256 tokens), and $45.1\times$ on longer sequences, while consistently maintaining higher accuracy than the baseline. Our method achieves significantly higher throughput ($6.8\times$ on GSM8K) than existing confidence-based approaches while preserving generation quality, enabling practical deployment of diffusion LLMs.
TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection
University of Science and Technology of China(中国科学技术大学);Tsinghua University(清华大学);Alibaba Cloud Computing(阿里云计算);Xiaohongshu Inc.(小红书公司);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Crystal-KV: Efficient KV Cache Management for Chain-of-Thought LLMs via Answer-First Principle
School of Computer Science and Technology, University of Science and Technology of China(计算机科学与技术学院,中国科学技术大学);University of Science and Technology of China(中国科学技术大学);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(人工智能研究所,合肥综合性国家科学中心);Suzhou Institute for Advanced Research, University of Science and Technology of China(苏州先进研究院,中国科学技术大学)
ZSMerge: Zero-Shot KV Cache Compression for Memory-Efficient Long-Context LLMs
Information Hub, The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学信息中心);School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院)
VIS Lab, University of Amsterdam(阿姆斯特丹大学VIS实验室);FunAI Lab, University of Technology Nuremberg(纽伦堡技术大学FunAI实验室);CSAIL, Massachusetts Institute of Technology(麻省理工学院计算机科学与人工智能实验室)
Cache-to-Cache: Direct Semantic Communication Between Large Language Models
Tsinghua University(清华大学);Infinigence AI;The Chinese University of Hong Kong(香港中文大学);Shanghai Jiao Tong University(上海交通大学);SLAI;Shanghai AI Laboratory(上海人工智能实验室)
KV-CoRE: Benchmarking Data-Dependent Low-Rank Compressibility of KV-Caches in LLMs
University at Buffalo(布法罗大学);Institute of Artificial Intelligence (TeleAI), China Telecom(人工智能研究所(TeleAI),中国电信);Delft University of Technology(代尔夫特理工大学);University of Maryland(马里兰大学);ByteDance(字节跳动)
WaveFilter: Enhancing the Long-Context Capability of Diffusion LLMs via Wavelet-Guided KV Cache Filtering
Nanjing University of Science and Technology(南京理工大学);Alibaba Group(阿里巴巴集团);Huzhou Normal University(湖州师范学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
KV Packet: Recomputation-Free Context-Independent KV Caching for LLMs
Technical University of Munich(慕尼黑技术大学);Technical University of Darmstadt(达姆施塔特技术大学);Zhejiang University(浙江大学);Technische Universität Ilmenau(伊尔梅瑙技术大学)
Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization
School of Computing and Information Systems, The University of Melbourne(墨尔本大学计算与信息系统学院);School of Computer Science and Technology, Huazhong University of Science and Technology(华中科技大学计算机科学与技术学院)
d$^2$Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education(新一代人工智能技术及其交叉应用国家重点实验室(东南大学));Southeast University(东南大学);Qiyuan Tech(启元科技)
SWAN: Sparse Winnowed Attention for Reduced Inference Memory via Decompression-Free KV-Cache Compression
Centre for Responsible AI(负责任人工智能中心);Indian Institute of Technology Madras(印度理工学院马德拉斯分校);Department of Electrical Engineering(电气工程系);Wadhwani School of Data Science and Artificial Intelligence(Wadhwani数据科学与人工智能学院)
PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling
University of Wisconsin - Madison(威斯康星大学麦迪逊分校);Peking University(北京大学);Nanjing University(南京大学);University of Surrey(Surrey 大学);Qwen(通义实验室);University of California - Riverside(加州大学河滨分校);Microsoft(微软公司)