子网络发现与软掩码用于混合任务的持续学习
Sub-network Discovery and Soft-masking for Continual Learning of Mixed Tasks
AI总结：
本文提出一种持续学习方法，通过发现子网络隔离任务知识以克服灾难性遗忘，并采用软掩码机制促进知识迁移，在混合任务上优于强基线。
摘要：
持续学习（CL）有两个主要目标：防止灾难性遗忘（CF）和促进知识迁移（KT）。现有文献主要关注克服CF。当任务相似时，也有一些工作致力于KT。据我们所知，只有一种方法被提出来学习混合任务序列。然而，这些技术仍然遭受CF和/或有限的KT。本文提出了一种新的CL方法来实现这两个目标。它通过为每个任务发现一个子网络来隔离每个任务的知识，从而克服CF。还提出了一种软掩码机制，以保留先前的知识，并使新任务能够利用过去的知识来实现KT。使用分类、生成、信息提取及其混合（即异构任务）的实验表明，所提出的方法始终优于强基线。
英文摘要：
Continual learning (CL) has two main objectives: preventing catastrophic forgetting (CF) and encouraging knowledge transfer (KT). The existing literature mainly focused on overcoming CF. Some work has also been done on KT when the tasks are similar. To our knowledge, only one method has been proposed to learn a sequence of mixed tasks. However, these techniques still suffer from CF and/or limited KT. This paper proposes a new CL method to achieve both. It overcomes CF by isolating the knowledge of each task via discovering a subnetwork for it. A soft-masking mechanism is also proposed to preserve the previous knowledge and to enable the new task to leverage the past knowledge to achieve KT. Experiments using classification, generation, information extraction, and their mixture (i.e., heterogeneous tasks) show that the proposed method consistently outperforms strong baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
