利用领域特定适配器与特征门控增强持续学习中的知识保留
Enhancing knowledge retention for continual learning with domain-specific adapters and features gating
AI总结：
本文提出在Vision Transformer自注意力中集成领域特定适配器和特征门控，以缓解持续学习中的灾难性遗忘，并通过多数据集实验证明策略性任务排序能进一步提升模型的知识保留与适应能力。
摘要：
持续学习使模型能够从连续的数据流中学习，同时保留先前获得的知识，从而有效应对灾难性遗忘的挑战。在本研究中，我们提出了一种新方法，将适配器集成到Vision Transformers的自注意力机制中，以在顺序添加来自不同领域的数据集时增强知识保留。与以往仅使用一个数据集继续学习的方法不同，我们的方法引入了领域特定的输出头和特征门控，使模型能够在保持先前学习任务高准确率的同时，仅纳入来自多个领域的基本信息。我们将所提出的方法与当前最先进的参数高效微调方法进行了比较。结果证明，我们的方法有效缓解了以往工作的局限性。此外，我们使用三个数据集CIFAR-100、Flowers102和DTD进行了比较分析，每个数据集代表一个不同的领域，以研究任务顺序对模型性能的影响。我们的研究结果强调了数据集排序在塑造学习结果中的关键作用，表明策略性排序可以显著提高模型随时间适应不断演变的数据分布的能力，同时保持先前学习知识的完整性。
英文摘要：
Continual learning empowers models to learn from a continuous stream of data while preserving previously acquired knowledge, effectively addressing the challenge of catastrophic forgetting. In this study, we propose a new approach that integrates adapters within the self-attention mechanisms of Vision Transformers to enhance knowledge retention when sequentially adding datasets from different domains. Unlike previous methods that continue learning with only one dataset, our approach introduces domain-specific output heads and feature gating, allowing the model to maintain high accuracy on previously learned tasks while incorporating only the essential information from multiple domains. The proposed method is compared to prominent parameter-efficient fine-tuning methods in the current state of the art. The results provide evidence that our method effectively alleviates the limitations of previous works. Furthermore, we conduct a comparative analysis using three datasets, CIFAR-100, Flowers102, and DTD, each representing a distinct domain, to investigate the impact of task order on model performance. Our findings underscore the critical role of dataset sequencing in shaping learning outcomes, demonstrating that strategic ordering can significantly improve the model's ability to adapt to evolving data distributions over time while preserving the integrity of previously learned knowledge.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
