PROL：通过提示在线学习实现流数据中的免排练持续学习
PROL : Rehearsal Free Continual Learning in Streaming Data via Prompt Online Learning
AI总结：
本文提出PROL方法，通过轻量级提示生成器、可训练缩放移位器、PTM泛化保持和硬-软更新机制，在流数据在线持续学习中实现免排练并超越现有SOTA性能。
摘要：
在线持续学习（OCL）中的数据隐私约束意味着数据只能被看到一次，这使得流数据中的灾难性遗忘问题更加复杂。当前OCL中的SOTA方法通常采用的一种做法是使用节省内存的样本或来自先前类别的特征在当前任务中进行重放。另一方面，基于提示的方法在持续学习中表现优异，但代价是可训练参数数量不断增长。第一种方法由于数据开放政策在实践中可能不适用，而第二种方法则存在与流数据相关的吞吐量问题。在本研究中，我们提出了一种新颖的基于提示的在线持续学习方法，包含4个主要组成部分：（1）作为通用知识的单一轻量级提示生成器，（2）作为特定知识的可训练缩放器与移位器，（3）预训练模型（PTM）泛化保持，以及（4）硬-软更新机制。我们提出的方法在CIFAR100、ImageNet-R、ImageNet-A和CUB数据集上取得了显著高于当前SOTA的性能。我们的复杂度分析表明，该方法需要相对较少的参数数量，并实现了适中的训练时间、推理时间和吞吐量。为了进一步研究，我们方法的源代码可在https://github.com/anwarmaxsum/PROL获取。
英文摘要：
The data privacy constraint in online continual learning (OCL), where the data can be seen only once, complicates the catastrophic forgetting problem in streaming data. A common approach applied by the current SOTAs in OCL is with the use of memory saving exemplars or features from previous classes to be replayed in the current task. On the other hand, the prompt-based approach performs excellently in continual learning but with the cost of a growing number of trainable parameters. The first approach may not be applicable in practice due to data openness policy, while the second approach has the issue of throughput associated with the streaming data. In this study, we propose a novel prompt-based method for online continual learning that includes 4 main components: (1) single light-weight prompt generator as a general knowledge, (2) trainable scaler-and-shifter as specific knowledge, (3) pre-trained model (PTM) generalization preserving, and (4) hard-soft updates mechanism. Our proposed method achieves significantly higher performance than the current SOTAs in CIFAR100, ImageNet-R, ImageNet-A, and CUB dataset. Our complexity analysis shows that our method requires a relatively smaller number of parameters and achieves moderate training time, inference time, and throughput. For further study, the source code of our method is available at https://github.com/anwarmaxsum/PROL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
