Cortex：将上下文整合到参数化记忆中
Cortex: Consolidating Context into Parametric Memory
AI总结：
针对大语言模型移除上下文后性能增益丢失的问题，提出Cortex方法，通过生成前瞻性查询蒸馏上下文信息，在多规模多模态模型上实现了良好的巩固-保留平衡，在14B模型的50次顺序事实编辑任务中表现优异。
摘要：
上下文可提升大语言模型性能，但移除上下文后这些增益可能丢失。受生物系统记忆巩固的启发，我们将记忆巩固（MC）定义为在保留非目标能力的同时内化这些增益，在统一目标下关联上下文指定编辑、技能学习与行为引导。我们推出含5个任务族的测试平台，用于测量单次、批量及顺序更新下的巩固与保留情况。我们提出Cortex，其生成前瞻性查询以预测上下文的用途，并将上下文条件输出蒸馏为可合并的低秩或结构化前馈网络更新。在4B至120B的密集型及混合专家主干（含纯文本与多模态模型）中，Cortex在多数评估场景中处于经验性巩固-保留帕累托前沿。在14B模型上进行50次顺序事实编辑后，Cortex-Structural恢复了88.9%的上下文诱导增益，同时保留了99.8%的原始非目标性能。
英文摘要：
Context can improve large language model performance, but these gains may be lost after context removal. Inspired by biological systems consolidation, we formulate Memory Consolidation (MC) as internalizing these gains while preserving non-target capabilities, connecting context-specified editing, skill learning, and behavioral steering under a common objective. We introduce a five-task-family testbed measuring consolidation and retention under single, batch, and sequential updates. We propose Cortex, which generates prospective queries to anticipate uses of context and distills context-conditioned outputs into mergeable low-rank or structural feed-forward network updates. Across 4B--120B dense and mixture-of-experts backbones, including both text-only and multimodal models, Cortex lies on the empirical consolidation--retention Pareto frontier in most evaluated settings. After 50 sequential factual edits on a 14B model, Cortex-Structural recovers 88.9% of the context-induced gain while retaining 99.8% of original non-target performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
