面向任务不均衡持续学习的动态锚定提示
Dynamically Anchored Prompting for Task-Imbalanced Continual Learning
AI总结：
针对任务不均衡持续学习场景，提出动态锚定提示（DAP）方法，通过维护单个通用提示并利用增强锚点和稳定锚点进行正则化，有效平衡稳定性与可塑性，无需排练即可取得显著性能提升。
摘要：
现有的持续学习文献严重依赖于任务伴随均衡数据流到达的强假设，这在现实应用中往往是不切实际的。在本研究中，我们探索了任务不均衡持续学习（TICL）场景，其中任务数据在整个学习过程中的分布是不均匀的。我们发现，从近期基于提示的持续学习方法的角度来看，不均衡的任务显著挑战了模型控制稳定性与可塑性之间权衡的能力。基于上述发现，我们提出了动态锚定提示（DAP），这是一种基于提示的方法，仅维护单个通用提示以动态适应任务流内的变化。该通用提示在提示空间中通过两个专门设计的提示锚点（称为增强锚点和稳定锚点）进行正则化，以在TICL中平衡稳定性和可塑性。值得注意的是，DAP通过在数据流中仅存储一个提示来实现这种平衡，因此在无排练持续学习中提供了显著优势。大量实验表明，在任务不均衡设置下的基准测试中，所提出的DAP相较于最先进的方法取得了4.5%至15%的绝对提升。我们的代码可在https://github.com/chenxing6666/DAP获取。
英文摘要：
Existing continual learning literature relies heavily on a strong assumption that tasks arrive with a balanced data stream, which is often unrealistic in real-world applications. In this work, we explore task-imbalanced continual learning (TICL) scenarios where the distribution of task data is non-uniform across the whole learning process. We find that imbalanced tasks significantly challenge the capability of models to control the trade-off between stability and plasticity from the perspective of recent prompt-based continual learning methods. On top of the above finding, we propose Dynamically Anchored Prompting (DAP), a prompt-based method that only maintains a single general prompt to adapt to the shifts within a task stream dynamically. This general prompt is regularized in the prompt space with two specifically designed prompt anchors, called boosting anchor and stabilizing anchor, to balance stability and plasticity in TICL. Remarkably, DAP achieves this balance by only storing a prompt across the data stream, therefore offering a substantial advantage in rehearsal-free CL. Extensive experiments demonstrate that the proposed DAP results in 4.5% to 15% absolute improvements over state-of-the-art methods on benchmarks under task-imbalanced settings. Our code is available at https://github.com/chenxing6666/DAP
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
