面向语言模型的无排练模块化与组合式持续学习
Rehearsal-Free Modular and Compositional Continual Learning for Language Models
AI总结：
本文提出无排练的模块化与组合式持续学习框架MoCL，通过持续添加并组合模块，在避免遗忘的同时促进任务间知识迁移，并在多个基准上超越现有方法。
摘要：
持续学习旨在增量获取新知识，同时不遗忘已有知识。为克服灾难性遗忘，现有方法要么基于排练，即存储先前任务的数据样本用于数据回放，要么为每个任务隔离专用参数。然而，基于排练的方法会引发隐私和内存问题，而参数隔离式持续学习未考虑任务之间的交互，从而阻碍了知识迁移。在本工作中，我们提出MoCL，一个无排练的模块化与组合式持续学习框架，该框架持续向语言模型添加新模块，并将其与已有模块进行组合。在多个基准上的实验表明，MoCL优于当前最先进方法，并能有效促进知识迁移。
英文摘要：
Continual learning aims at incrementally acquiring new knowledge while not forgetting existing knowledge. To overcome catastrophic forgetting, methods are either rehearsal-based, i.e., store data examples from previous tasks for data replay, or isolate parameters dedicated to each task. However, rehearsal-based methods raise privacy and memory issues, and parameter-isolation continual learning does not consider interaction between tasks, thus hindering knowledge transfer. In this work, we propose MoCL, a rehearsal-free Modular and Compositional Continual Learning framework which continually adds new modules to language models and composes them with existing modules. Experiments on various benchmarks show that MoCL outperforms state of the art and effectively facilitates knowledge transfer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
