基于动态模块扩展与适应的终身序列生成
Lifelong Sequence Generation with Dynamic Module Expansion and Adaptation
AI总结：
针对终身序列生成中跨任务知识迁移不足及灾难性遗忘问题，提出DMEA方法，通过动态决定模型架构并利用相似历史任务促进适应，结合动态梯度缩放平衡学习，在多种设置下持续优于现有方法。
摘要：
终身序列生成（LSG）是持续学习中的一个问题，旨在连续训练模型处理一系列生成任务，以学习不断涌现的新生成模式，同时避免遗忘先前知识。现有LSG方法主要侧重于维持旧知识，却很少关注跨任务的知识迁移。相比之下，人类能够利用从相似任务中已获取的知识来更好地学习新任务。受人类学习范式启发，我们提出了动态模块扩展与适应（DMEA），该机制使模型能够基于任务相关性动态决定获取新知识的架构，并选择最相似的先前任务以促进对新任务的适应。此外，由于学习过程容易偏向当前任务，可能导致对先前所学知识更严重的遗忘，我们提出了动态梯度缩放来平衡当前任务与重放任务的学习。通过大量实验，我们证明DMEA能在不同LSG设置中持续优于现有方法。
英文摘要：
Lifelong sequence generation (LSG), a problem in continual learning, aims to continually train a model on a sequence of generation tasks to learn constantly emerging new generation patterns while avoiding the forgetting of previous knowledge. Existing LSG methods mainly focus on maintaining old knowledge while paying little attention to knowledge transfer across tasks. In contrast, humans can better learn new tasks by leveraging previously acquired knowledge from similar tasks. Inspired by the learning paradigm of humans, we propose Dynamic Module Expansion and Adaptation (DMEA), which enables the model to dynamically determine the architecture for acquiring new knowledge based on task correlation and select the most similar previous tasks to facilitate adaptation to new tasks. In addition, as the learning process can easily be biased towards the current task which might cause more severe forgetting of previously learned knowledge, we propose dynamic gradient scaling to balance the learning of the current task and replayed tasks. With extensive experiments, we demonstrate that DMEA can consistently outperform existing methods in different LSG settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
