终身学习过程：自我记忆监控与动态增长网络
Lifelong Learning Process: Self-Memory Supervising and Dynamically Growing Networks
摘要：
从童年到青年，人类逐渐认识世界。但对于神经网络来说，这个成长过程似乎很困难。陷入灾难性遗忘，目前的研究人员将所有类别的数据馈送到一个神经网络，该网络在整个训练过程中保持相同的结构。我们将这种训练过程与人类的学习模式进行了比较，发现了两个主要的冲突。本文基于条件变分自编码器（CVAE）模型，研究如何在生成模型上解决这些冲突。为了解决不连续的冲突，我们采用记忆回放策略来保持模型对不可见使用类别的识别和生成能力。并将传统的单向CVAE扩展为循环模式，以更好地实现内存回放策略。为了解决“死”结构的冲突，我们重写了CVAE公式，从而能够对CVAE模型中不同部分的功能做出新的解释。基于新的理解，我们找到了在新类别上训练时动态扩展网络结构的方法。我们验证了我们的方法的有效性MNIST和时尚MNIST和显示一些非常interesting结果。
英文摘要：
From childhood to youth, human gradually come to know the world. But for neural networks, this growing process seems difficult. Trapped in catastrophic forgetting, current researchers feed data of all categories to a neural network which remains the same structure in the whole training process. We compare this training process with human learing patterns, and find two major conflicts. In this paper, we study how to solve these conflicts on generative models based on the conditional variational autoencoder(CVAE) model. To solve the uncontinuous conflict, we apply memory playback strategy to maintain the model's recognizing and generating ability on invisible used categories. And we extend the traditional one-way CVAE to a circulatory mode to better accomplish memory playback strategy. To solve the `dead' structure conflict, we rewrite the CVAE formula then are able to make a novel interpretation about the funtions of different parts in CVAE models. Based on the new understanding, we find ways to dynamically extend the network structure when training on new categories. We verify the effectiveness of our methods on MNIST and Fashion MNIST and display some very insteresting results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
