基于贝叶斯原理的持续学习
Learning to Continually Learn with the Bayesian Principle
AI总结：
本文提出一种元持续学习框架，结合神经网络表示能力与统计模型贝叶斯更新的抗遗忘性，实现高性能且可扩展的持续学习。
摘要：
在当今深度学习时代，持续学习研究主要关注在使用随机梯度下降对非平稳数据流训练神经网络时减轻遗忘。另一方面，在更经典的统计机器学习文献中，许多模型具有顺序贝叶斯更新规则，这些规则产生与批量训练相同的学习结果，即它们完全不受灾难性遗忘的影响。然而，这些模型往往过于简单，无法建模复杂的现实世界数据。在这项工作中，我们采用元学习范式，结合神经网络的强大表示能力和简单统计模型对遗忘的鲁棒性。在我们新颖的元持续学习框架中，持续学习仅通过理想的顺序贝叶斯更新规则在统计模型中进行，而神经网络则被元学习以桥接原始数据和统计模型。由于神经网络在持续学习期间保持固定，它们免受灾难性遗忘的影响。这种方法不仅实现了显著改进的性能，而且表现出出色的可扩展性。由于我们的方法是领域无关和模型无关的，它可以应用于广泛的问题，并轻松与现有模型架构集成。
英文摘要：
In the present era of deep learning, continual learning research is mainly focused on mitigating forgetting when training a neural network with stochastic gradient descent on a non-stationary stream of data. On the other hand, in the more classical literature of statistical machine learning, many models have sequential Bayesian update rules that yield the same learning outcome as the batch training, i.e., they are completely immune to catastrophic forgetting. However, they are often overly simple to model complex real-world data. In this work, we adopt the meta-learning paradigm to combine the strong representational power of neural networks and simple statistical models' robustness to forgetting. In our novel meta-continual learning framework, continual learning takes place only in statistical models via ideal sequential Bayesian update rules, while neural networks are meta-learned to bridge the raw data and the statistical models. Since the neural networks remain fixed during continual learning, they are protected from catastrophic forgetting. This approach not only achieves significantly improved performance but also exhibits excellent scalability. Since our approach is domain-agnostic and model-agnostic, it can be applied to a wide range of problems and easily integrated with existing model architectures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
