无隔离学习：持续学习中的通路保护
Learning without Isolation: Pathway Protection for Continual Learning
AI总结：
本文提出无隔离学习（LwI）框架，将模型融合视为图匹配，通过保护旧任务占用的通路而非参数，利用激活通道稀疏性自适应分配通路，以参数高效方式解决持续学习中的灾难性遗忘问题。
摘要：
深度网络在顺序任务学习过程中容易遭受灾难性遗忘，即在学习新任务时丢失关于旧任务的知识。为此，持续学习（CL）应运而生，其现有方法大多侧重于调节或保护与先前任务相关的参数。然而，参数保护往往不切实际，因为存储旧任务知识所需的参数规模随任务数量线性增长，否则难以保留与旧任务知识相关的参数。在这项工作中，我们从神经科学和物理学中引入双重观点到持续学习：在整个网络中，关于从旧任务获取的知识，通路比参数更为重要。遵循这一观点，我们提出了一种新颖的持续学习框架——无隔离学习（LwI），其中模型融合被表述为图匹配，旧任务所占用的通路得到保护而不被隔离。得益于深度网络中激活通道的稀疏性，LwI能够为新任务自适应地分配可用通路，实现通路保护并以参数高效的方式解决灾难性遗忘问题。在流行基准数据集上的实验证明了所提出的LwI的优越性。
英文摘要：
Deep networks are prone to catastrophic forgetting during sequential task learning, i.e., losing the knowledge about old tasks upon learning new tasks. To this end, continual learning(CL) has emerged, whose existing methods focus mostly on regulating or protecting the parameters associated with the previous tasks. However, parameter protection is often impractical, since the size of parameters for storing the old-task knowledge increases linearly with the number of tasks, otherwise it is hard to preserve the parameters related to the old-task knowledge. In this work, we bring a dual opinion from neuroscience and physics to CL: in the whole networks, the pathways matter more than the parameters when concerning the knowledge acquired from the old tasks. Following this opinion, we propose a novel CL framework, learning without isolation(LwI), where model fusion is formulated as graph matching and the pathways occupied by the old tasks are protected without being isolated. Thanks to the sparsity of activation channels in a deep network, LwI can adaptively allocate available pathways for a new task, realizing pathway protection and addressing catastrophic forgetting in a parameter-efficient manner. Experiments on popular benchmark datasets demonstrate the superiority of the proposed LwI.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
