发表机构
Université de Montréal; Mila Quebec AI Institute; University of Oregon; McGill University（蒙特利尔大学; 魁北克人工智能研究所米拉; 俄勒冈大学; 麦吉尔大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
GaTaP通过门控变量与双时间尺度学习，在持续学习中保持旧任务性能并实现少样本组合泛化。
AI 中文摘要
持续学习通常被定义为在不灾难性遗忘先前任务的情况下获取新知识。然而，一个灵活的持续学习者还应能够重用并重新组合先前获得的知识，以快速解决新颖的任务组合。我们引入了门控目标传播（GaTaP），一种持续学习算法，其中通过闭式内循环更新学习的任务特定门控变量选择性地抑制或增强网络模块。网络参数在较慢时间尺度的外循环中学习，使用与适应门控变量相同的局部差异目标传播误差信号。我们在多层感知机和卷积网络架构的类增量学习场景中提供了可处理的实验。我们展示了在先前学习任务上的强性能保持，以及通过对未见任务的少样本增益自适应实现的组合泛化。我们分析了学习到的门控模式，发现相关任务表现出相似的门控模式，表明推断的门控捕获了有意义、可重用的任务结构。总体而言，GaTaP提供了一个强大的框架，可同时改善灾难性遗忘并在神经网络模型中实现少样本组合泛化。
英文摘要
Continual learning is typically framed as acquiring new knowledge without catastrophically forgetting previous tasks. However, a flexible continual learner should also be able to reuse and recombine previously acquired knowledge to rapidly solve novel task compositions. We introduce Gated Target Propagation (GaTaP), a continual learning algorithm in which task-specific gating variables---learned through a closed-form inner loop update---selectively suppress or enhance network modules. Network parameters are learned in a slower timescale outer loop, using the same local difference target propagation error signal as is used for adapting gating variables. We provide tractable experiments on class-incremental learning scenarios for both multilayer perceptron and convolutional network architectures. We show strong performance retention on previously learned tasks, as well as compositional generalization to unseen tasks, achieved through few-shot gain adaptation at inference. We analyze learned gating patterns and find that related tasks exhibit similar gating patterns, suggesting that inferred gates capture meaningful, reusable task structure. Overall, GaTaP provides a powerful framework for jointly ameliorating catastrophic forgetting and enabling few-shot compositional generalization in neural network models.
