基于贝叶斯自适应矩正则化的鲁棒持续学习
Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization
AI总结：
本文提出贝叶斯自适应矩正则化（BAdam）方法，通过更好地约束参数增长来减少灾难性遗忘，在Split MNIST等基准上实现基于先验方法的最先进性能，且无需任务标签或离散边界。
摘要：
追求长期自主性要求机器学习模型必须不断适应变化的环境并学习解决新任务。持续学习旨在克服灾难性遗忘的挑战，即学习新任务会导致模型遗忘先前学到的信息。基于先验的持续学习方法因其计算效率高且不需要辅助模型或数据存储而备受青睐。然而，基于先验的方法在重要基准上通常表现不佳，因此与基于记忆的方法相比，其潜在应用受到限制。我们提出了贝叶斯自适应矩正则化（BAdam），这是一种新颖的基于先验的方法，能更好地约束参数增长，减少灾难性遗忘。该方法具有轻量级、无需任务标签、快速收敛等优点，并提供对安全现实部署至关重要的校准不确定性。结果表明，BAdam在Split MNIST和Split FashionMNIST等具有挑战性的单头类增量实验中，实现了基于先验方法的最先进性能，且无需依赖任务标签或离散任务边界。
英文摘要：
The pursuit of long-term autonomy mandates that machine learning models must continuously adapt to their changing environments and learn to solve new tasks. Continual learning seeks to overcome the challenge of catastrophic forgetting, where learning to solve new tasks causes a model to forget previously learnt information. Prior-based continual learning methods are appealing as they are computationally efficient and do not require auxiliary models or data storage. However, prior-based approaches typically fail on important benchmarks and are thus limited in their potential applications compared to their memory-based counterparts. We introduce Bayesian adaptive moment regularization (BAdam), a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting. Our method boasts a range of desirable properties such as being lightweight and task label-free, converging quickly, and offering calibrated uncertainty that is important for safe real-world deployment. Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments such as Split MNIST and Split FashionMNIST, and does so without relying on task labels or discrete task boundaries.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
