MEMENTO:教大模型管理自身上下文
MEMENTO: Teaching LLMs to Manage Their Own Context
AI总结:
MEMENTO通过将推理分块并压缩为密集状态摘要,减少上下文、KV缓存和计算开销,提升模型效率和准确性。
摘要:
推理模型通常以长而无结构的流形式思考,缺乏压缩或组织自身中间状态的机制。我们引入MEMENTO:一种方法,教模型将推理分为块,将每个块压缩为密集状态摘要(即memento),并通过仅关注mementos进行推理,从而减少上下文、KV缓存和计算。为了训练MEMENTO模型,我们发布了OpenMementos,一个包含228,000个推理轨迹的公开数据集,源自OpenThoughts-v3,已分割并注释了中间摘要。我们展示了在OpenMementos上使用两阶段SFT配方在不同模型家族(Qwen3、Phi-4、Olmo 3)和规模(8B-32B参数)上均有效。训练后的模型在数学、科学和编码基准测试中保持强劲的准确性,同时实现约2.5倍的峰值KV缓存减少。我们扩展了vLLM以支持我们的推理方法,实现约1.75倍的吞吐量提升,同时使我们能够进行强化学习并进一步提高准确性。最后,我们识别出双信息流:每个推理块的信息既通过memento文本传递,也通过相应的KV状态保留,这些状态保留了原始块的隐含信息。移除这一通道会使AIME24的准确性下降15个百分点。
英文摘要:
Reasoning models think in long, unstructured streams with no mechanism for compressing or organizing their own intermediate state. We introduce MEMENTO: a method that teaches models to segment reasoning into blocks, compress each block into a memento, i.e., a dense state summary, and reason forward by attending only to mementos, reducing context, KV cache, and compute. To train MEMENTO models, we release OpenMementos, a public dataset of 228K reasoning traces derived from OpenThoughts-v3, segmented and annotated with intermediate summaries. We show that a two-stage SFT recipe on OpenMementos is effective across different model families (Qwen3, Phi-4, Olmo 3) and scales (8B--32B parameters). Trained models maintain strong accuracy on math, science, and coding benchmarks while achieving ${\sim}2.5\times$ peak KV cache reduction. We extend vLLM to support our inference method, achieving ${\sim}1.75\times$ throughput improvement while also enabling us to perform RL and further improve accuracy. Finally, we identify a dual information stream: information from each reasoning block is carried both by the memento text and by the corresponding KV states, which retain implicit information from the original block. Removing this channel drops accuracy by 15\,pp on AIME24.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。