EvolKV：面向LLM推理的进化式KV缓存压缩
EvolKV: Evolutionary KV Cache Compression for LLM Inference
AI总结：
EvolKV提出一种基于进化搜索的自适应KV缓存压缩框架，通过多目标优化逐层分配缓存预算，在11个任务上超越基线，并以1.5%预算在代码补全上超越全缓存。
摘要：
现有的键值（KV）缓存压缩方法通常依赖于启发式策略，例如在各层之间均匀分配缓存或采用静态驱逐策略，然而，这些方法忽略了层特定特征模式与任务性能之间的关键相互作用，这可能导致泛化性能下降。在本文中，我们提出了EvolKV，一个自适应的逐层、任务驱动的KV缓存压缩框架，它联合优化了内存效率和任务性能。通过将缓存分配重新表述为多目标优化问题，EvolKV利用进化搜索动态配置层预算，同时直接最大化下游性能。在11个任务上的大量实验表明，我们的方法在长上下文任务的各种KV缓存预算下均优于所有基线方法，并在GSM8K上比启发式基线高出最多7个百分点。值得注意的是，EvolKV在仅使用原始预算的1.5%的情况下，在代码补全任务上实现了优于完整KV缓存设置的性能，这表明在KV缓存预算分配的学习压缩策略中存在着尚未开发的潜力。
英文摘要：
Existing key-value (KV) cache compression methods typically rely on heuristics, such as uniform cache allocation across layers or static eviction policies, however, they ignore the critical interplays among layer-specific feature patterns and task performance, which can lead to degraded generalization. In this paper, we propose EvolKV, an adaptive framework for layer-wise, task-driven KV cache compression that jointly optimizes the memory efficiency and task performance. By reformulating cache allocation as a multi-objective optimization problem, EvolKV leverages evolutionary search to dynamically configure layer budgets while directly maximizing downstream performance. Extensive experiments on 11 tasks demonstrate that our approach outperforms all baseline methods across a wide range of KV cache budgets on long-context tasks and surpasses heuristic baselines by up to 7 percentage points on GSM8K. Notably, EvolKV achieves superior performance over the full KV cache setting on code completion while utilizing only 1.5% of the original budget, suggesting the untapped potential in learned compression strategies for KV cache budget allocation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
