LatentRemedy：多智能体系统中非前缀KV缓存复用的高效补救方案
LatentRemedy: Efficient Remedy For Non-Prefix KV Cache Reuse in Multi-Agent Systems
AI总结：
LatentRemedy通过任务条件潜层令牌解决多智能体系统冲突任务下非前缀KV缓存复用的性能下降问题，在保持接近直接复用的TTFT的同时优于现有基线。
摘要：
非前缀KV缓存复用是提升多智能体系统推理效率的有前景方法，因为智能体可共享相同的上下文块。然而，当智能体任务存在冲突时，跨智能体直接复用非前缀KV缓存会严重降低模型性能，导致性能下降高达90%。现有的补救措施包括选择性重计算和基于偏移的校准，仅能恢复有限比例的性能损失。我们认为该问题不仅在于受污染的KV表示，还在于智能体难以在自身指令下解释复用的上下文。为此，我们提出LatentRemedy，其利用少量任务条件潜层令牌帮助目标智能体复用上下文，无需重计算共享块。具体而言，我们使用MLP将目标前缀的最后隐藏状态映射为少量潜层令牌，这些令牌被放置在复用KV块之后以与共享上下文交互。为引导后续预测朝向目标任务，我们通过全预填蒸馏训练该MLP。实验表明，在冲突任务上，LatentRemedy始终优于现有的复用基线，且随着缓存规模增长，其首令牌生成时间（TTFT）接近直接复用的水平。代码和数据可在该https URL获取。
英文摘要：
Non-prefix KV cache reuse is a promising approach for improving inference efficiency in multi-agent systems as agents share the same context blocks. However, when agents have conflicting tasks, directly reusing non-prefix KV caches across agents can severely degrade model performance, causing up to a 90% drop. Existing remedies, including selective recomputation and offset-based calibration, recover a limited fraction of this loss. We argue that the problem lies not only in contaminated KV representations but also in the agent's difficulty interpreting reused context under its own instructions. To this end, we introduce LatentRemedy, which uses a few task-conditioned latent tokens to help the target agent leverage reused context without recomputing the shared block. Specifically, we use an MLP to map the last hidden state of the target prefix into a small number of latent tokens, which are placed after the reused KV block to interact with the shared context. To guide subsequent predictions toward the target task, we train the MLP through full-prefill distillation. Experiments show that LatentRemedy consistently outperforms existing reuse baselines on conflicting tasks while maintaining TTFT close to direct reuse as cache size grows. Code and data are available at https://github.com/chenzhwsysu57/kvreuse.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
