分数衰减键值缓存：面向对话系统提升推理相关性的所有权感知内存管理
Fractional Decay KV-Cache: Ownership-Aware Memory Management for Improved Inference Relevancy in Dialog Systems
浏览论文内容
中文总结 AI 辅助
提出FD-KVC算法，通过双通道评分机制实现对话系统KV缓存的所有权感知内存管理，在多轮对话场景中多项指标优于H2O，适配新话题更快且话题多样性更高。
中文摘要 AI 辅助
键值（KV）缓存是基于Transformer的对话系统高效自回归推理的关键，但现有策略要么对所有缓存条目一视同仁，要么采用粗粒度的淘汰启发式方法，无法随对话话题演变而自适应调整。我们提出分数衰减键值缓存（Fractional Decay KV-Cache，FD-KVC），这是一种新型算法，为每个缓存的KV对维护双通道评分机制：跟踪聚合重要性的累积注意力通道（类似于H2O），以及由时间衰减和强化启发式更新控制的近期加权相关性通道。二者的结合使FD-KVC既能保留历史重要token，又能在对话话题转变时快速适配。由所有权损失函数驱动的自适应学习率确保收敛而无振荡。FD-KVC完全在CPU上运行，开销可忽略。在5种不同的多轮对话场景（每种含600个对话）中，FD-KVC在复合后轮对齐指标上比当前最先进的重量级基准H2O提升6.7%，在话题转变指标上提升127%，在渐进演变指标上提升87%，在混合话题对话指标上提升30%；FD-KVC适配新话题的速度是H2O的3.6倍，且在所有方法中达到最高的话题多样性（80.6%）。消融研究证实了每个组件的贡献。
英文摘要
Key-value (KV) caching is essential for efficient autoregressive inference in transformer based dialog systems, yet existing strategies treat all cached entries uniformly or apply coarse eviction heuristics that fail to adapt as dialog topics evolve. We propose Fractional Decay KV-Cache (FD-KVC), a novel algorithm that maintains a dual-channel scoring mechanism for each cached KV pair: a cumulative attention channel that tracks aggregate importance (akin to H2O), and a recency-weighted relevance channel governed by temporal decay and reinforcement-inspired updates. The combination enables FD-KVC to both preserve historically important tokens and rapidly adapt when dialog topics shift. An adaptive learning rate driven by an ownership loss function ensures convergence without oscillation. FD-KVC operates entirely on CPU with negligible overhead. Across five diverse multi-turn dialog scenarios with 600 dialogs each, FD-KVC outperforms H2O, the state-of-the-art heavy-hitter baseline, by +6.7% on composite late-turn alignment, with improvements of +127% on topic-shift, +87% on gradual evolution, and +30% on mixed-topic dialogs. FD-KVC adapts to new topics 3.6X faster than H2O and achieves the highest topic diversity (80.6%) across all methods. Ablation studies confirm the contribution of each component.
发表机构
- NetApp Inc（NetApp公司）
机构由 AI 辅助整理，请以论文原文为准。
