在线持续学习中的快速适应：我们的评估方式正确吗？
Rapid Adaptation in Online Continual Learning: Are We Evaluating It Right?
AI总结：
本文指出在线准确率指标因虚假标签相关性而不可靠，提出基于近未来样本准确率的新指标，发现保留和重用历史信息能实现更好的泛化。
摘要：
我们重新审视了通过在线准确率指标评估在线持续学习（OCL）算法适应性的常见做法，该指标衡量模型在紧接着的接下来几个样本上的准确率。然而，我们表明该指标是不可靠的，因为即使是预测时不使用输入图像的空洞盲分类器，也能通过利用数据流中虚假的标签相关性来实现高得不切实际的在线准确率。我们的研究揭示，现有的OCL算法也能达到很高的在线准确率，但在保留有用信息方面表现不佳，这表明它们无意中学习了虚假的标签相关性。为了解决这个问题，我们提出了一种基于近未来样本准确率来衡量适应性的新指标，其中虚假相关性已被消除。我们在各种计算预算下使用我们提出的指标在大型数据集上对现有的OCL方法进行了基准测试，发现通过保留和重用过去见过的信息可以实现更好的泛化能力。我们相信我们提出的指标能够帮助开发真正具有适应性的OCL方法。我们在 https://github.com/drimpossible/EvalOCL 提供了复现结果的代码。
英文摘要：
We revisit the common practice of evaluating adaptation of Online Continual Learning (OCL) algorithms through the metric of online accuracy, which measures the accuracy of the model on the immediate next few samples. However, we show that this metric is unreliable, as even vacuous blind classifiers, which do not use input images for prediction, can achieve unrealistically high online accuracy by exploiting spurious label correlations in the data stream. Our study reveals that existing OCL algorithms can also achieve high online accuracy, but perform poorly in retaining useful information, suggesting that they unintentionally learn spurious label correlations. To address this issue, we propose a novel metric for measuring adaptation based on the accuracy on the near-future samples, where spurious correlations are removed. We benchmark existing OCL approaches using our proposed metric on large-scale datasets under various computational budgets and find that better generalization can be achieved by retaining and reusing past seen information. We believe that our proposed metric can aid in the development of truly adaptive OCL methods. We provide code to reproduce our results at https://github.com/drimpossible/EvalOCL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
