无需持续训练的持续学习

中文摘要 AI 辅助

持续学习要求模型在适应新领域和新类别的同时保留先验知识。许多现有方法依赖于持续优化，使用正则化、重放或参数扩展来防止新更新覆盖先前学到的知识。相反，我们提出用持续推理取代持续训练：一个基于PFN的模型，经过元训练后冻结，仅通过扩展上下文证据集来适应新类别。我们的模型，即潜在概念PFN，在捕获跨领域和类别共享的语义结构的潜在概念空间上执行上下文贝叶斯推理。当每个新领域或类别到来时，示例被添加到记忆中；适应反映的是对潜在概念的后验信念的更新，而非梯度更新。没有参数被改变，从而减少了遗忘。同一方法无需任务身份即可处理领域增量和类别增量持续学习。概念注释仅在元训练期间使用，作为潜在空间上的软锚点，而非固定瓶颈。与固定词汇概念方法不同，该模型通过将概念标签与原始输入证据相结合，还能处理噪声、模糊或不完整的注释，从而发现超出预定义概念集的区分。在类别和领域增量学习数据集上的实验表明，该模型在实现可解释潜在概念学习的同时，展现了具有竞争力的持续学习性能。

英文摘要

Continual learning requires models to adapt to new domains and new classes while retaining prior knowledge. Many existing methods rely on continued optimization, using regularization, replay, or parameter expansion to prevent new updates from overwriting previously learned knowledge. Instead, we propose replacing continual training with continual inference: a PFN-based model that is meta-trained, and then frozen, adapting to new classes only by extending an in-context evidence set. Our model, Latent Concept PFN, performs in-context Bayesian inference over a latent concept space that captures semantic structure shared across domains and classes. As each new domain or class arrives, exemplars are added to the memory; adaptation reflects updated posterior beliefs over latent concepts rather than gradient updates. No parameters are changed, reducing forgetting. The same method handles both domain and class incremental continual learning without task identity. Concept annotations are only used during meta-training, acting as a soft anchor on the latent space rather than a fixed bottleneck. Unlike fixed-vocabulary concept methods, the model also handles noisy, ambiguous, or incomplete annotations by combining concept labels with raw input evidence to discover distinctions beyond the predefined concept set. Experiments on class and domain incremental learning datasets demonstrate competitive continual learning performance while learning interpretable latent concepts.

发表机构

• Imperial College London（帝国理工学院）

机构由 AI 辅助整理，请以论文原文为准。