分析持续学习者的新指标
New metrics for analyzing continual learners
AI总结：
针对持续学习中现有指标忽视任务难度增加的问题，提出考虑难度增加的新指标，并通过基准实验揭示稳定性-可塑性权衡的新见解。
摘要：
深度神经网络在从固定类别集合的独立同分布数据上训练时已展现出卓越的性能。然而，在现实世界场景中，可能期望在连续的数据流上训练模型，其中多个分类任务按顺序呈现。这种场景被称为持续学习（CL），它对标准学习算法提出了挑战，这些算法在努力学习新任务的同时难以保持对旧任务的知识。这种稳定性-可塑性困境仍然是持续学习的核心，并且已经提出了多种指标来分别充分衡量稳定性和可塑性。然而，没有一个指标考虑到分类任务难度的增加，这固有地导致任何模型的性能损失。就此而言，我们分析了当前指标的一些局限性，并识别出由设置引起的遗忘的存在。因此，我们提出了考虑任务难度增加的新指标。通过在基准数据集上的实验，我们证明了所提出的指标能够为持续学习环境中模型所实现的稳定性-可塑性权衡提供新的见解。
英文摘要：
Deep neural networks have shown remarkable performance when trained on independent and identically distributed data from a fixed set of classes. However, in real-world scenarios, it can be desirable to train models on a continuous stream of data where multiple classification tasks are presented sequentially. This scenario, known as Continual Learning (CL) poses challenges to standard learning algorithms which struggle to maintain knowledge of old tasks while learning new ones. This stability-plasticity dilemma remains central to CL and multiple metrics have been proposed to adequately measure stability and plasticity separately. However, none considers the increasing difficulty of the classification task, which inherently results in performance loss for any model. In that sense, we analyze some limitations of current metrics and identify the presence of setup-induced forgetting. Therefore, we propose new metrics that account for the task's increasing difficulty. Through experiments on benchmark datasets, we demonstrate that our proposed metrics can provide new insights into the stability-plasticity trade-off achieved by models in the continual learning environment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
