增量深度神经网络中灾难性遗忘的方法论导向研究
A Methodology-Oriented Study of Catastrophic Forgetting in Incremental Deep Neural Networks
AI总结：
该综述针对增量深度神经网络的灾难性遗忘问题，聚焦评估机制相似的算法，对比三类增量学习方法，提供高影响力方法的数学概述以助力相关研究。
摘要：
人类和不同种类的动物具备在一生中收集、迁移知识、处理、微调并生成信息的能力。这种终身学习的能力被称为持续学习，它依赖神经认知机制实现。因此，在现实世界的增量学习自主智能体计算系统中，也需要这种能够实现信息检索和长期记忆巩固的持续学习机制。然而，人工智能领域的主要挑战在于自主智能体面对新数据时的增量学习问题。在这类场景中，核心问题是灾难性遗忘（catastrophic forgetting, CF），即神经网络在按顺序学习时，接触新数据后会对旧数据出现欠拟合。为解决灾难性遗忘问题，已有大量研究被提出，但由于评估机制存在差异，很难对它们的性能进行比较。本文聚焦于对比所有采用相似类型评估机制的算法，将对比三类增量学习方法：（1）基于样本（Exemplar）的方法，（2）基于记忆的方法，（3）基于网络的方法。本综述论文针对增量深度神经网络中的灾难性遗忘问题开展了方法论导向的研究，此外还包含了对高影响力方法的数学概述，可帮助研究人员应对灾难性遗忘问题。
英文摘要：
Human being and different species of animals having the skills to gather, transferring knowledge, processing, fine-tune and generating information throughout their lifetime. The ability of learning throughout their lifespan is referred as continuous learning which is using neurocognition mechanism. Consequently, in real world computational system of incremental learning autonomous agents also needs such continuous learning mechanism which provide retrieval of information and long-term memory consolidation. However, the main challenge in artificial intelligence is that the incremental learning of the autonomous agent when new data confronted. In such scenarios, the main concern is catastrophic forgetting(CF), i.e., while learning the sequentially, neural network underfits the old data when it confronted with new data. To tackle this CF problem many numerous studied have been proposed, however it is very difficult to compare their performance due to dissimilarity in their evaluation mechanism. Here we focus on the comparison of all algorithms which are having similar type of evaluation mechanism. Here we are comparing three types of incremental learning methods: (1) Exemplar based methods, (2) Memory based methods, and (3) Network based method. In this survey paper, methodology oriented study for catastrophic forgetting in incremental deep neural network is addressed. Furthermore, it contains the mathematical overview of impact-full methods which can be help researchers to deal with CF.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
