NestedKV: 用于长上下文KV缓存压缩的嵌套内存路由
NestedKV: Nested Memory Routing for Long-Context KV Cache Compression
AI总结：
提出NestedKV方法，通过多时间尺度余弦异常评分和头自适应混合的免训练键缓存压缩，在长上下文任务中优于现有方法。
摘要：
长上下文语言模型受限于键值（KV）缓存的内存占用。现有的免训练KV压缩方法通常通过单一重要性信号（注意力、近期性、逐层分配或键独特性）对令牌进行排序，这在有用上下文具有全局独特性、局部片段性或即时相关性时变得脆弱。我们引入NestedKV，一种受嵌套学习中连续内存系统启发的仅键KV缓存压缩方法。NestedKV维护全局、块级和滑动窗口键锚点，通过多时间尺度余弦异常对令牌评分，并将所得排名与使用头自适应混合和惊喜门控令牌路由的免训练外部学习器结合。该评分与自适应每头预算配对，无需训练或修改LLM。在RULER（4k--32k）、LooGLE、LongBench、LongBench-E、InfiniteBench和MMLU-Pro上，使用Qwen3和Llama-3.2模型，NestedKV在保留缓存较小时表现最强。在Qwen3-4B上，当r=0.75时，它在RULER上比KeyDiff提升高达19.10分，在LongBench上提升19.29分；当r=0.95时，它在LongBench上保留37.32分，而KeyDiff为17.55分。
英文摘要：
Long-context language models are limited by the memory footprint of the key-value (KV) cache. Existing training-free KV compression methods usually rank tokens by one importance signal -- attention, recency, layer-wise allocation, or key distinctiveness -- which becomes brittle when useful context is globally distinctive, locally episodic, or immediately relevant. We introduce NestedKV, a key-only KV cache compression method inspired by the Continuum Memory System in Nested Learning. NestedKV maintains global, block-level, and sliding-window key anchors, scores tokens by multi-time-scale cosine anomaly, and combines the resulting rankings with a training-free outer learner using head-adaptive mixing and surprise-gated token routing. The score is paired with adaptive per-head budgets and requires no training or LLM modification. Across RULER (4k--32k), LooGLE, LongBench, LongBench-E, InfiniteBench, and MMLU-Pro on Qwen3 and Llama-3.2 models, NestedKV is strongest when the retained cache is small. On Qwen3-4B, it improves over KeyDiff by up to 19.10 points on RULER and 19.29 on LongBench at $r=0.75$; at $r=0.95$, it retains 37.32 on LongBench versus 17.55 for KeyDiff.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
