arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.00487cs.LGcs.AIcs.CV
收藏

持续学习综合综述:理论、方法与应​​用

A Comprehensive Survey of Continual Learning: Theory, Method and Application

Tsinghua University(清华大学)

Liyuan Wang, Xingxing Zhang, Hang Su, Jun Zhu

AI总结:

本文全面综述持续学习,涵盖理论、方法与实际应用,总结其目标为平衡稳定性-可塑性及泛化性,并提供最新分类体系以促进后续研究。

摘要:

为应对现实世界的动态变化,智能系统需要在其生命周期内增量地获取、更新、积累和利用知识。这种能力被称为持续学习,它为人工智能系统自适应地发展奠定了基础。从一般意义上讲,持续学习明确受到灾难性遗忘的限制,即学习新任务通常会导致旧任务性能的急剧下降。除此之外,近年来出现了越来越多的进展,在很大程度上扩展了对持续学习的理解和应用。对这一方向的兴趣日益增长且广泛,表明了其现实意义和复杂性。在这项工作中,我们对持续学习进行了全面的综述,旨在弥合基本设置、理论基础、代表性方法和实际应用之间的鸿沟。基于现有的理论和实证结果,我们将持续学习的总体目标总结为:在资源效率的背景下,确保适当的稳定性-可塑性权衡以及足够的任务内/任务间泛化能力。然后,我们提供了一个最先进且详尽分类的体系,深入分析了代表性方法如何解决持续学习问题,以及它们如何适应实际应用中的特定挑战。通过对有前景方向的深入讨论,我们相信这种整体视角可以极大地促进该领域及其后续的探索。

英文摘要:

To cope with real-world dynamics, an intelligent system needs to incrementally acquire, update, accumulate, and exploit knowledge throughout its lifetime. This ability, known as continual learning, provides a foundation for AI systems to develop themselves adaptively. In a general sense, continual learning is explicitly limited by catastrophic forgetting, where learning a new task usually results in a dramatic performance degradation of the old tasks. Beyond this, increasingly numerous advances have emerged in recent years that largely extend the understanding and application of continual learning. The growing and widespread interest in this direction demonstrates its realistic significance as well as complexity. In this work, we present a comprehensive survey of continual learning, seeking to bridge the basic settings, theoretical foundations, representative methods, and practical applications. Based on existing theoretical and empirical results, we summarize the general objectives of continual learning as ensuring a proper stability-plasticity trade-off and an adequate intra/inter-task generalizability in the context of resource efficiency. Then we provide a state-of-the-art and elaborated taxonomy, extensively analyzing how representative methods address continual learning, and how they are adapted to particular challenges in realistic applications. Through an in-depth discussion of promising directions, we believe that such a holistic perspective can greatly facilitate subsequent exploration in this field and beyond.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑