arXiv 2605.124842026-05-14cs.LGcs.AI

学习，快速与缓慢：迈向能够持续适应的LLM

Learning, Fast and Slow: Towards LLMs That Adapt Continually

UC Berkeley（加州大学伯克利分校）；Mila（蒙特利尔大学人工智能研究所）；UT Austin（得克萨斯大学奥斯汀分校）；Eragon Periodic Labs；Mirendil Video

Rishabh Tiwari, Kusha Sareen, Lakshya A Agrawal, Joseph E. Gonzalez, Matei Zaharia, Kurt Keutzer, Inderjit S Dhillon, Rishabh Agarwal, Devvrit Khatri

备注：: 29 pages, 14 figures, including appendix; Blog post: https://gepa-ai.github.io/gepa/blog/2026/05/11/learning-fast-and-slow/

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出一种快速-缓慢学习框架，通过将模型参数作为缓慢权重和优化上下文作为快速权重，提升LLM在连续学习中的样本效率和性能，减少灾难性遗忘。

摘要：

大型语言模型（LLMs）通过更新参数（如通过强化学习）来训练下游任务，但这种更新迫使它们吸收特定任务信息，导致灾难性遗忘和失去可塑性。相比之下，固定LLM参数的上下文学习可以廉价快速适应任务需求（如提示优化），但无法自身匹配通过更新参数获得的性能提升。人类也可能以不同的时间尺度学习（如系统1与系统2）。为此，我们引入了一种快速-缓慢学习框架，将模型参数作为“缓慢”权重，优化上下文作为“快速”权重。这些快速“权重”可以从文本反馈中学习以吸收任务特定信息，同时允许缓慢权重保持接近基础模型并保留一般推理行为。快速-缓慢训练（FST）在推理任务中比仅缓慢学习（强化学习）高效3倍，同时在性能渐进上达到更高的水平。此外，FST训练的模型更接近基础LLM（KL散度减少多达70%），导致比强化学习训练更少的灾难性遗忘。这种减少的漂移也保留了可塑性：在训练一个任务后，FST训练的模型在后续任务中适应更有效，而仅参数训练的模型则不然。在连续学习场景中，当任务领域实时变化时，FST继续获取每个新任务，而仅参数强化学习则停滞。

英文摘要：

Large language models (LLMs) are trained for downstream tasks by updating their parameters (e.g., via RL). However, updating parameters forces them to absorb task-specific information, which can result in catastrophic forgetting and loss of plasticity. In contrast, in-context learning with fixed LLM parameters can cheaply and rapidly adapt to task-specific requirements (e.g., prompt optimization), but cannot by itself typically match the performance gains available through updating LLM parameters. There is no good reason for restricting learning to being in-context or in-weights. Moreover, humans also likely learn at different time scales (e.g., System 1 vs 2). To this end, we introduce a fast-slow learning framework for LLMs, with model parameters as "slow" weights and optimized context as "fast" weights. These fast "weights" can learn from textual feedback to absorb the task-specific information, while allowing slow weights to stay closer to the base model and persist general reasoning behaviors. Fast-Slow Training (FST) is up to 3x more sample-efficient than only slow learning (RL) across reasoning tasks, while consistently reaching a higher performance asymptote. Moreover, FST-trained models remain closer to the base LLM (up to 70% less KL divergence), resulting in less catastrophic forgetting than RL-training. This reduced drift also preserves plasticity: after training on one task, FST trained models adapt more effectively to a subsequent task than parameter-only trained models. In continual learning scenarios, where task domains change on the fly, FST continues to acquire each new task while parameter-only RL stalls.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。