链接适配器：连接过去与未来以有效实现持续学习
Linked Adapters: Linking Past and Future to Present for Effective Continual Learning
AI总结：
提出链接适配器方法，利用MLP建模的加权注意力机制在任务适配器间传递知识，克服灾难性遗忘，并在图像分类数据集上验证了性能提升。
摘要：
持续学习使系统能够学习并适应新任务，同时保留从先前任务中获得的知识。然而，深度学习模型在学习新任务时，会遭受对早期任务所学知识的灾难性遗忘。此外，为每个新任务从头重新训练像Transformer这样的大型模型成本高昂。解决持续学习的一个有效方法是使用带有任务特定适配器的大型预训练模型来适应新任务。尽管这种方法可以缓解灾难性遗忘，但由于每个任务分别学习适配器，它们无法跨任务传递知识。为解决此问题，我们提出了一种新颖的方法——链接适配器，它通过加权注意力机制实现向其他任务特定适配器的知识传递。链接适配器使用多层感知器（MLP）来建模注意力权重，这除了建模前向知识传递外，还克服了持续学习中后向知识传递的挑战。在推理过程中，我们提出的方法通过基于MLP的注意力权重，有效利用所有横向任务适配器之间的知识传递。通过在多种图像分类数据集上进行的大量实验，我们有效证明了使用链接适配器在持续学习任务上的性能提升。
英文摘要：
Continual learning allows the system to learn and adapt to new tasks while retaining the knowledge acquired from previous tasks. However, deep learning models suffer from catastrophic forgetting of knowledge learned from earlier tasks while learning a new task. Moreover, retraining large models like transformers from scratch for every new task is costly. An effective approach to address continual learning is to use a large pre-trained model with task-specific adapters to adapt to the new tasks. Though this approach can mitigate catastrophic forgetting, they fail to transfer knowledge across tasks as each task is learning adapters separately. To address this, we propose a novel approach Linked Adapters that allows knowledge transfer through a weighted attention mechanism to other task-specific adapters. Linked adapters use a multi-layer perceptron (MLP) to model the attention weights, which overcomes the challenge of backward knowledge transfer in continual learning in addition to modeling the forward knowledge transfer. During inference, our proposed approach effectively leverages knowledge transfer through MLP-based attention weights across all the lateral task adapters. Through numerous experiments conducted on diverse image classification datasets, we effectively demonstrated the improvement in performance on the continual learning tasks using Linked Adapters.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
