今天的持续学习算法有多高效？
How Efficient Are Today's Continual Learning Algorithms?
AI总结：
本文研究了当前持续学习算法在计算、内存和存储方面的效率问题，发现许多增量类学习方法效率极低，甚至比从零开始训练消耗更多计算资源，强调持续学习不应仅关注缓解灾难性遗忘，还需重视资源效率以实现现实适用性。
摘要：
监督式持续学习涉及从不断增长的带标签数据流中更新深度神经网络（DNN）。尽管大多数工作集中在克服灾难性遗忘，但持续学习背后的主要动机之一是能够高效地用新信息更新网络，而不是随着训练数据集随时间增长而从零开始重新训练。尽管最近的持续学习方法在很大程度上解决了灾难性遗忘问题，但这些算法的效率却鲜少受到关注。在这里，我们研究了增量类学习的最新方法，并说明许多方法在计算、内存和存储方面效率极低。有些方法甚至需要比从零开始训练更多的计算量！我们认为，要使持续学习具有现实适用性，研究界不能忽视这些算法所使用的资源。持续学习不仅仅是缓解灾难性遗忘。
英文摘要：
Supervised Continual learning involves updating a deep neural network (DNN) from an ever-growing stream of labeled data. While most work has focused on overcoming catastrophic forgetting, one of the major motivations behind continual learning is being able to efficiently update a network with new information, rather than retraining from scratch on the training dataset as it grows over time. Despite recent continual learning methods largely solving the catastrophic forgetting problem, there has been little attention paid to the efficiency of these algorithms. Here, we study recent methods for incremental class learning and illustrate that many are highly inefficient in terms of compute, memory, and storage. Some methods even require more compute than training from scratch! We argue that for continual learning to have real-world applicability, the research community cannot ignore the resources used by these algorithms. There is more to continual learning than mitigating catastrophic forgetting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
