序列化以缓解持续学习中的灾难性遗忘
Sequencing to Mitigate Catastrophic Forgetting in Continual Learning
AI总结：
本文提出通过最优任务序列缓解持续学习中的灾难性遗忘，结合零样本评分算法并增强性能与鲁棒性。
摘要：
为应对现实世界中的动态变化，智能系统需要在其生命周期内逐步获取、更新和利用知识。这种能力称为持续学习，为AI系统自身适应性发展提供了基础。灾难性遗忘是持续学习方法进展的主要挑战，其中学习新任务通常会导致对之前学习任务的性能大幅下降。许多方法已出现以对抗灾难性遗忘的影响。大多数提出的方案可以分为五类：重放类、正则化类、优化类、表示类和架构类。在本工作中，我们从不同的角度出发，具体考虑任务在呈现给模型时的最优序列。我们探讨了任务序列在缓解灾难性遗忘中的作用，并提出了一种确定最优任务顺序的方法。所提出的方法利用受神经架构搜索（NAS）启发的零样本评分算法。结果表明，智能的任务序列可以显著减少灾难性遗忘。此外，当与传统持续学习策略结合时，序列化提供了增强的性能和对遗忘的鲁棒性。此外，所提出的方法还可应用于其他领域，如课程学习。
英文摘要：
To cope with real-world dynamics, an intelligent system needs to incrementally acquire, update, and exploit knowledge throughout its lifetime. This ability, known as Continual learning, provides a foundation for AI systems to develop themselves adaptively. Catastrophic forgetting is a major challenge to the progress of Continual Learning approaches, where learning a new task usually results in a dramatic performance drop on previously learned ones. Many approaches have emerged to counteract the impact of CF. Most of the proposed approaches can be categorized into five classes: replay-based, regularization-based, optimization-based, representation-based, and architecture-based. In this work, we approach the problem from a different angle, specifically by considering the optimal sequencing of tasks as they are presented to the model. We investigate the role of task sequencing in mitigating CF and propose a method for determining the optimal task order. The proposed method leverages zero-shot scoring algorithms inspired by neural architecture search (NAS). Results demonstrate that intelligent task sequencing can substantially reduce CF. Moreover, when combined with traditional continual learning strategies, sequencing offers enhanced performance and robustness against forgetting. Additionally, the presented approaches can find applications in other fields, such as curriculum learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
