关于持续学习的序列最大后验推断
On Sequential Maximum a Posteriori Inference for Continual Learning
AI总结：
本文将持续学习转化为对先前损失函数的近似问题，提出自动微分二次巩固和神经巩固两种无核心集方法，并在经典与图像任务序列中验证了各自的有效性。
摘要：
我们将序列最大后验推断形式化为损失函数的递归，并将持续学习问题归结为对先前损失函数的近似。随后，我们提出两种无核心集方法：autodiff quadratic consolidation（自动微分二次巩固），它使用精确且完整的二次近似；以及 neural consolidation（神经巩固），它使用神经网络近似。这些方法在神经网络规模方面不具备可扩展性，因此我们结合固定的预训练特征提取器，针对分类任务对它们进行研究。我们还引入了基于 Iris 和 Wine 数据集的简单但具有挑战性的经典任务序列。我们发现，neural consolidation 在输入维度较小的经典任务序列中表现良好，而 autodiff quadratic consolidation 在具有固定预训练特征提取器的图像任务序列中表现始终良好，在许多情况下达到了与联合最大后验训练相当的性能。
英文摘要：
We formulate sequential maximum a posteriori inference as a recursion of loss functions and reduce the problem of continual learning to approximating the previous loss function. We then propose two coreset-free methods: autodiff quadratic consolidation, which uses an accurate and full quadratic approximation, and neural consolidation, which uses a neural network approximation. These methods are not scalable with respect to the neural network size, and we study them for classification tasks in combination with a fixed pre-trained feature extractor. We also introduce simple but challenging classical task sequences based on Iris and Wine datasets. We find that neural consolidation performs well in the classical task sequences, where the input dimension is small, while autodiff quadratic consolidation performs consistently well in image task sequences with a fixed pre-trained feature extractor, achieving comparable performance to joint maximum a posteriori training in many cases.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
