ATLAS：基于适配器的多模态持续学习与两阶段学习策略
ATLAS: Adapter-Based Multi-Modal Continual Learning with a Two-Stage Learning Strategy
AI总结：
本文提出ATLAS，一种基于适配器的多模态持续学习两阶段范式，通过经验学习与新知识扩展缓解灾难性遗忘，提升下游任务泛化能力。
摘要：
尽管视觉与语言模型在许多领域取得了显著进展，但持续学习的挑战仍未解决。适配器和提示等参数高效模块为缓解灾难性遗忘提供了一种有前景的方法。然而，现有工作通常为每个任务学习单独的适配器，这可能导致适配器之间存在冗余知识。此外，它们继续使用原始预训练模型来初始化下游模型，导致与原始模型相比，模型的泛化能力变化微乎其微。另外，目前仍缺乏研究探讨将多模态模型整合到单模态和多模态任务的更新过程中所带来的后果，及其对下游任务的后续影响。在本文中，我们提出了一种基于适配器的两阶段学习范式，这是一种多模态持续学习方案，包含基于经验的学习和新知识扩展，帮助模型充分利用经验知识并补偿新知识。大量实验表明，我们的方法在持续学习方面表现出色。它在扩展上游表示分布的同时，也最小化了遗忘先前任务的负面影响。此外，它增强了下游任务的泛化能力。进一步地，我们将多模态和单模态任务都纳入上游持续学习。我们观察到，从上游任务学习可以帮助下游任务。我们的代码将在以下地址提供：https://github.com/lihong2303/ATLAS。
英文摘要：
While vision-and-language models significantly advance in many fields, the challenge of continual learning is unsolved. Parameter-efficient modules like adapters and prompts present a promising way to alleviate catastrophic forgetting. However, existing works usually learn individual adapters for each task, which may result in redundant knowledge among adapters. Moreover, they continue to use the original pre-trained model to initialize the downstream model, leading to negligible changes in the model's generalization compared to the original model. In addition, there is still a lack of research investigating the consequences of integrating a multi-modal model into the updating procedure for both uni-modal and multi-modal tasks and the subsequent impacts it has on downstream tasks. In this paper, we propose an adapter-based two-stage learning paradigm, a multi-modal continual learning scheme that consists of experience-based learning and novel knowledge expansion, which helps the model fully use experience knowledge and compensate for novel knowledge. Extensive experiments demonstrate that our method is proficient for continual learning. It expands the distribution of representation upstream while also minimizing the negative impact of forgetting previous tasks. Additionally, it enhances the generalization capability for downstream tasks. Furthermore, we incorporate both multi-modal and uni-modal tasks into upstream continual learning. We observe that learning from upstream tasks can help with downstream tasks. Our code will be available at: https://github.com/lihong2303/ATLAS.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
