KOPPA：基于键-查询正交投影与基于原型的“一对多”方法改进基于提示的持续学习
KOPPA: Improving Prompt-based Continual Learning with Key-Query Orthogonal Projection and Prototype-based One-Versus-All
AI总结：
本文提出KOPPA方法，通过正交投影键-查询学习和OVA原型组件改进基于提示的持续学习，在基准数据集上超越现有方法达20%。
摘要：
受应用于大语言模型的提示调优技术启发，近期基于预训练ViT网络的方法在持续学习领域取得了显著成果。具体而言，这些方法提出维护一组提示，并通过键-查询匹配策略分配其中一部分来学习每个任务。然而，当缺乏对旧任务查询与未来任务键之间相关性的控制、潜在空间中特征发生偏移以及独立任务中学习的潜在向量相对分离不足时，这些方法可能会遇到局限。在本工作中，我们引入了一种受模型无关元学习启发的基于正交投影的新型键-查询学习策略，以提高提示匹配效率并应对特征偏移的挑战。此外，我们引入了一个基于“一对多”（OVA）原型的组件，以增强分类头的区分能力。在基准数据集上的实验结果表明，我们的方法使模型能够取得超越当前最先进方法的结果，且提升幅度高达20%。
英文摘要：
Drawing inspiration from prompt tuning techniques applied to Large Language Models, recent methods based on pre-trained ViT networks have achieved remarkable results in the field of Continual Learning. Specifically, these approaches propose to maintain a set of prompts and allocate a subset of them to learn each task using a key-query matching strategy. However, they may encounter limitations when lacking control over the correlations between old task queries and keys of future tasks, the shift of features in the latent space, and the relative separation of latent vectors learned in independent tasks. In this work, we introduce a novel key-query learning strategy based on orthogonal projection, inspired by model-agnostic meta-learning, to enhance prompt matching efficiency and address the challenge of shifting features. Furthermore, we introduce a One-Versus-All (OVA) prototype-based component that enhances the classification head distinction. Experimental results on benchmark datasets demonstrate that our method empowers the model to achieve results surpassing those of current state-of-the-art approaches by a large margin of up to 20%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
