基于强经验回放的持续学习
Continual Learning with Strong Experience Replay
AI总结：
本文提出强经验回放（SER）持续学习方法，通过在当前训练数据上模拟未来经验并结合过去经验蒸馏，提升新旧模型预测一致性，在多个图像分类数据集上显著超越现有方法。
摘要：
持续学习（CL）旨在增量式地学习新任务，同时不遗忘从旧任务中获得的知识。经验回放（ER）是一种简单而有效的基于排练的策略，它利用当前训练数据和存储在记忆缓冲区中的旧样本子集来优化模型。为了进一步减少遗忘，近期方法通过模型正则化和记忆采样等多种技术对ER进行了扩展。然而，新模型与旧模型在当前训练数据上的预测一致性很少被探索，导致在可用旧样本很少时保留的知识较少。为了解决这一问题，我们提出了一种具有强经验回放（SER）的持续学习方法，该方法除了从记忆缓冲区中蒸馏过去经验之外，还额外利用在当前训练数据上模拟的未来经验。在我们的方法中，更新后的模型将产生与其原始输出近似的输出，从而能够有效保留已获得的知识。在多个图像分类数据集上的实验结果表明，我们的SER方法以明显优势超越了当前最先进的方法。
英文摘要：
Continual Learning (CL) aims at incrementally learning new tasks without forgetting the knowledge acquired from old ones. Experience Replay (ER) is a simple and effective rehearsal-based strategy, which optimizes the model with current training data and a subset of old samples stored in a memory buffer. To further reduce forgetting, recent approaches extend ER with various techniques, such as model regularization and memory sampling. However, the prediction consistency between the new model and the old one on current training data has been seldom explored, resulting in less knowledge preserved when few previous samples are available. To address this issue, we propose a CL method with Strong Experience Replay (SER), which additionally utilizes future experiences mimicked on the current training data, besides distilling past experience from the memory buffer. In our method, the updated model will produce approximate outputs as its original ones, which can effectively preserve the acquired knowledge. Experimental results on multiple image classification datasets show that our SER method surpasses the state-of-the-art methods by a noticeable margin.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
