将持续学习重塑为序列建模
Recasting Continual Learning as Sequence Modeling
AI总结：
本文将持续学习表述为序列建模问题，利用元持续学习（MCL）框架在元层级训练序列模型，并验证了Transformers及其高效变体在七个基准上作为通用MCL方法的有效性。
摘要：
在本研究中，我们旨在建立两大量化机器学习研究体系之间的重要联系：持续学习与序列建模。具体而言，我们提出将持续学习表述为序列建模问题，从而允许将先进的序列模型应用于持续学习。在这种表述下，持续学习过程转变为序列模型的前向传递。通过采用元持续学习（MCL）框架，我们能够在元层级上，基于多个持续学习片段来训练序列模型。作为新表述的具体示例，我们展示了Transformers及其高效变体作为MCL方法的应用。我们在七个基准上的实验涵盖了分类和回归任务，结果表明序列模型可以成为通用MCL的一种极具吸引力的解决方案。
英文摘要：
In this work, we aim to establish a strong connection between two significant bodies of machine learning research: continual learning and sequence modeling. That is, we propose to formulate continual learning as a sequence modeling problem, allowing advanced sequence models to be utilized for continual learning. Under this formulation, the continual learning process becomes the forward pass of a sequence model. By adopting the meta-continual learning (MCL) framework, we can train the sequence model at the meta-level, on multiple continual learning episodes. As a specific example of our new formulation, we demonstrate the application of Transformers and their efficient variants as MCL methods. Our experiments on seven benchmarks, covering both classification and regression, show that sequence models can be an attractive solution for general MCL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
