基础模型时代持续学习的未来：三个关键方向
The Future of Continual Learning in the Era of Foundation Models: Three Key Directions
AI总结：
本文论证在基础模型时代持续学习仍不可替代，并提出持续预训练、持续微调和持续组合性三个关键方向，其中持续组合性将推动持续学习的重生。
摘要：
持续学习——即随时间获取、保留和精炼知识的能力——一直是人类智能和人工智能的基础。历史上，不同的人工智能范式都承认这一需求，但侧重点各异：早期专家系统和生产系统关注增量知识整合，而强化学习强调动态适应。随着深度学习的兴起，深度持续学习主要关注随时间学习鲁棒且可复用的表示，以解决一系列日益复杂的任务。然而，大语言模型（LLMs）和基础模型的出现提出了一个问题：当集中式、单体式模型能够借助互联网规模的知识处理多样化任务时，我们是否还需要持续学习？我们认为，持续学习仍然至关重要，原因有三：（i）持续预训练仍然是必要的，以确保基础模型保持最新状态，缓解知识陈旧和分布偏移，同时整合新信息；（ii）持续微调使模型能够专业化和个性化，适应领域特定任务、用户偏好和现实约束，而无需完全重新训练，避免了对计算成本高昂的长上下文窗口的需求；（iii）持续组合性提供了一种可扩展且模块化的智能方法，使基础模型和智能体的编排能够被动态组合、重组和适应。虽然持续预训练和持续微调被作为小众研究方向进行探索，但我们认为，正是持续组合性将标志着持续学习的重生。人工智能的未来不会由单一静态模型定义，而将由一个持续演化并相互作用的模型生态系统定义，这使得持续学习比以往任何时候都更加重要。
英文摘要：
Continual learning--the ability to acquire, retain, and refine knowledge over time--has always been fundamental to intelligence, both human and artificial. Historically, different AI paradigms have acknowledged this need, albeit with varying priorities: early expert and production systems focused on incremental knowledge consolidation, while reinforcement learning emphasised dynamic adaptation. With the rise of deep learning, deep continual learning has primarily focused on learning robust and reusable representations over time to solve sequences of increasingly complex tasks. However, the emergence of Large Language Models (LLMs) and foundation models has raised the question: Do we still need continual learning when centralised, monolithic models can tackle diverse tasks with access to internet-scale knowledge? We argue that continual learning remains essential for three key reasons: (i) continual pre-training is still necessary to ensure foundation models remain up to date, mitigating knowledge staleness and distribution shifts while integrating new information; (ii) continual fine-tuning enables models to specialise and personalise, adapting to domain-specific tasks, user preferences, and real-world constraints without full retraining, avoiding the need for computationally expensive long context-windows; (iii) continual compositionality offers a scalable and modular approach to intelligence, enabling the orchestration of foundation models and agents to be dynamically composed, recombined, and adapted. While continual pre-training and fine-tuning are explored as niche research directions, we argue it is continual compositionality that will mark the rebirth of continual learning. The future of AI will not be defined by a single static model but by an ecosystem of continually evolving and interacting models, making continual learning more relevant than ever.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
