痕迹推理：通过压缩信标实现的内存高效推理
Breadcrumbs Reasoning: Memory-Efficient Reasoning with Compression Beacons
AI总结：
本研究提出通过压缩信标实现内存高效推理，利用联合蒸馏和强化学习框架优化缓存压缩，提升大语言模型在长上下文推理中的内存与准确性平衡。
摘要：
大型语言模型在长上下文推理中的可扩展性严重受限于其Transformer键值缓存的线性增长，这带来了显著的内存和计算成本。我们提出，随着模型生成推理令牌，过去生成的令牌的信息价值会降低，这为压缩提供了机会。在本工作中，我们提出定期用一个学习得到的专用令牌压缩生成的KV缓存，并将压缩条目驱逐。我们通过修改的联合蒸馏和强化学习（RL）框架训练模型执行此压缩。我们的训练方法在传统RL过程中最小化了开销，因为它利用RL输出进行蒸馏。实验表明，我们的方法在内存-准确性帕累托前沿方面优于没有缓存压缩的模型和无训练压缩技术。
英文摘要：
The scalability of large language models for long-context reasoning is severely constrained by the linear growth of their Transformer key-value cache, which incurs significant memory and computational costs. We posit that as a model generates reasoning tokens, the informational value of past generated tokens diminishes, creating an opportunity for compression. In this work, we propose to periodically compress the generation KV cache with a learned, special-purpose token and evict compressed entries. We train the model to perform this compression via a modified joint distillation and reinforcement learning (RL) framework. Our training method minimizes overhead over the conventional RL process, as it leverages RL outputs for distillation. Empirically, our method achieves a superior memory-accuracy Pareto frontier compared to both the model without cache compression and training-free compression techniques.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
