I2I：利用即兴知识初始化适配器
I2I: Initializing Adapters with Improvised Knowledge
AI总结：
I2I通过从旧任务适配器蒸馏知识来初始化新任务适配器，在CLiMB视觉问答序列上以零额外参数成本超越独立训练和AdapterFusion，实现更优的跨任务知识迁移。
摘要：
适配器为持续学习中的灾难性遗忘问题提供了一种有前景的解决方案。然而，为每个新任务训练独立的适配器模块会错失跨任务知识迁移的机会。我们提出即兴初始化（Improvise to Initialize，I2I），这是一种持续学习算法，通过从先前学习任务的适配器中蒸馏知识来为即将到来的任务初始化适配器。我们在多模态持续学习基准 CLiMB 上评估 I2I，通过在视觉问答任务序列上进行实验。使用 I2I 训练的适配器始终比独立训练的适配器获得更好的任务准确率，表明我们的算法促进了任务适配器之间的知识迁移。与最先进的 AdapterFusion 相比，I2I 还实现了更好的跨任务知识迁移，且不产生相关的参数成本。
英文摘要：
Adapters present a promising solution to the catastrophic forgetting problem in continual learning. However, training independent Adapter modules for every new task misses an opportunity for cross-task knowledge transfer. We propose Improvise to Initialize (I2I), a continual learning algorithm that initializes Adapters for incoming tasks by distilling knowledge from previously-learned tasks' Adapters. We evaluate I2I on CLiMB, a multimodal continual learning benchmark, by conducting experiments on sequences of visual question answering tasks. Adapters trained with I2I consistently achieve better task accuracy than independently-trained Adapters, demonstrating that our algorithm facilitates knowledge transfer between task Adapters. I2I also results in better cross-task knowledge transfer than the state-of-the-art AdapterFusion without incurring the associated parametric cost.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
