PromptFusion：为持续学习解耦稳定性与可塑性
PromptFusion: Decoupling Stability and Plasticity for Continual Learning
AI总结：
针对持续学习的稳定性-可塑性困境，提出基于提示调优的PromptFusion方法，通过双模块分别处理遗忘与新知识学习，并推出轻量化版本，在多个数据集上取得优异性能。
摘要：
当前持续学习研究主要聚焦于缓解灾难性遗忘，且大多数研究的成功是以限制新输入任务的性能为代价的。这种权衡被称为稳定性-可塑性困境，是持续学习中一个更具普遍性与挑战性的问题。然而，这两个概念之间的内在冲突使得人们似乎不可能同时为两者设计出令人满意的解决方案。因此我们提出疑问：“是否有可能将它们拆分为两个独立的问题来分别攻克？” 为此，我们提出了一种基于提示调优（prompt-tuning）的方法，名为PromptFusion，以实现稳定性与可塑性的解耦。具体而言，PromptFusion包含一个精心设计的stab模块（用于应对灾难性遗忘）和一个boo模块（用于同时学习新知识）。此外，为解决额外架构带来的计算开销问题，我们提出了PromptFusion-Lite，它通过动态判断每张输入图像是否需要同时激活两个模块，对PromptFusion进行了改进。大量实验表明，在主流持续学习数据集的类别增量和领域增量设置下，PromptFusion与PromptFusion-Lite均取得了可观的结果。尤其是在类别增量学习最具挑战性的数据集之一Split-Imagenet-R上，我们的方法准确率超过现有最优的基于提示的方法5%以上，且PromptFusion-Lite的计算资源消耗比PromptFusion低14.8%。
英文摘要：
Current research on continual learning mainly focuses on relieving catastrophic forgetting, and most of their success is at the cost of limiting the performance of newly incoming tasks. Such a trade-off is referred to as the stability-plasticity dilemma and is a more general and challenging problem for continual learning. However, the inherent conflict between these two concepts makes it seemingly impossible to devise a satisfactory solution to both of them simultaneously. Therefore, we ask, "is it possible to divide them into two separate problems to conquer them independently?". To this end, we propose a prompt-tuning-based method termed PromptFusion to enable the decoupling of stability and plasticity. Specifically, PromptFusion consists of a carefully designed \stab module that deals with catastrophic forgetting and a \boo module to learn new knowledge concurrently. Furthermore, to address the computational overhead brought by the additional architecture, we propose PromptFusion-Lite which improves PromptFusion by dynamically determining whether to activate both modules for each input image. Extensive experiments show that both PromptFusion and PromptFusion-Lite achieve promising results on popular continual learning datasets for class-incremental and domain-incremental settings. Especially on Split-Imagenet-R, one of the most challenging datasets for class-incremental learning, our method can exceed state-of-the-art prompt-based methods by more than 5\% in accuracy, with PromptFusion-Lite using 14.8\% less computational resources than PromptFusion.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
