上下文学习能够像人类一样进行持续学习
In-Context Learning can Perform Continual Learning Like Humans
AI总结：
本文提出上下文持续学习 ICCL，通过任务调度与提示重排实现无参数更新的持续适应，并揭示类似人类的分散练习间隔效应及更贴近人类记忆模式的线性注意力模型。
摘要：
大型语言模型（LLM）能够通过上下文学习（ICL）在不更新参数的情况下适应新任务，这使其成为快速适应的强大学习引擎。尽管已有大量研究将 ICL 作为少样本学习器加以考察，但当多个任务依次到来时，它能否实现长期记忆保持和跨任务知识积累，仍未得到充分研究。受人类记忆研究启发，我们研究了多任务环境中 ICL 的记忆保持特征，并将其扩展为上下文持续学习（ICCL）；在 ICCL 中，持续学习能力通过任务调度和提示重排而涌现。在 Markov-Chain 基准上的实验表明，对于特定的大型语言模型，ICCL 以类似于人类的方式受益于分散练习（DP），并持续揭示出有利于记忆保持的间隔“最佳点”。除记忆保持性能外，我们提出了一种人类记忆保持相似性指标，用于量化持续学习（CL）方法与人类记忆保持动态的契合程度。利用该指标，我们表明 MAMBA 和 RWKV 等线性注意力模型表现出尤其接近人类的记忆保持模式，尽管其记忆保持性能落后于基于 Transformer 的 LLM。总体而言，我们的结果确立了 ICCL 在认知上的合理性和实践上的有效性，提供了一种仅需推理的 CL 范式，可缓解灾难性遗忘并解决传统 CL 方法中的稳定性—可塑性困境。
英文摘要：
Large language models (LLMs) can adapt to new tasks via in-context learning (ICL) without parameter updates, making them powerful learning engines for fast adaptation. While extensive research has examined ICL as a few-shot learner, whether it can achieve long-term retention and cross-task knowledge accumulation when multitasks arrive sequentially remains underexplored. Motivated by human memory studies, we investigate the retention characteristics of ICL in multitask settings and extend it to in-context continual learning (ICCL), where continual learning ability emerges through task scheduling and prompt rearrangement. Experiments on Markov-Chain benchmarks demonstrate that, for specific large-language models, ICCL benefits from distributed practice (DP) in a manner analogous to humans, consistently revealing a spacing "sweet spot" for retention. Beyond retention performance, we propose a human-retention similarity metric to quantify how closely a continual-learning (CL) method aligns with human retention dynamics. Using this metric, we show that linear-attention models such as MAMBA and RWKV exhibit particularly human-like retention patterns, despite their retention performance lagging behind that of Transformer-based LLMs. Overall, our results establish ICCL as both cognitively plausible and practically effective, providing an inference-only CL paradigm that mitigates catastrophic forgetting and addresses the stability-plasticity dilemma in conventional CL methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
