MGSER-SAM：结合锐度感知优化的记忆引导软经验回放以增强持续学习
MGSER-SAM: Memory-Guided Soft Experience Replay with Sharpness-Aware Optimization for Enhanced Continual Learning
AI总结：
针对持续学习中的灾难性遗忘问题，提出结合SAM优化器、软logits与记忆梯度对齐的MGSER-SAM算法，在三类场景下均优于现有基线，显著提升准确率并降低遗忘率。
摘要：
深度神经网络在持续学习（CL）领域存在灾难性遗忘问题。为应对这一挑战，我们提出MGSER-SAM，这是一种新型的基于记忆回放的算法，专门用于增强持续学习模型的泛化能力。我们首先整合了SAM优化器（一种用于优化平坦度的组件），它可无缝适配ER、DER++等知名经验回放框架。随后，MGSER-SAM独特地解决了当前任务与先前存储记忆之间权重扰动方向的冲突协调这一复杂挑战，该问题在SAM优化器的研究中尚未得到充分探索。这一目标通过策略性整合软logits与记忆梯度方向对齐得以有效实现，其中的正则化项有助于同时最小化持续学习过程中各类关键的训练损失项。通过在多个基准上开展的严格实验分析，MGSER-SAM已展现出在所有三种持续学习场景下始终优于现有基线的能力。与具有代表性的基于记忆回放的基线ER和DER++相比，MGSER-SAM不仅将测试准确率分别提升了24.4%和17.6%，还在每个基准上实现了最低的遗忘率。
英文摘要：
Deep neural networks suffer from the catastrophic forgetting problem in the field of continual learning (CL). To address this challenge, we propose MGSER-SAM, a novel memory replay-based algorithm specifically engineered to enhance the generalization capabilities of CL models. We first intergrate the SAM optimizer, a component designed for optimizing flatness, which seamlessly fits into well-known Experience Replay frameworks such as ER and DER++. Then, MGSER-SAM distinctively addresses the complex challenge of reconciling conflicts in weight perturbation directions between ongoing tasks and previously stored memories, which is underexplored in the SAM optimizer. This is effectively accomplished by the strategic integration of soft logits and the alignment of memory gradient directions, where the regularization terms facilitate the concurrent minimization of various training loss terms integral to the CL process. Through rigorous experimental analysis conducted across multiple benchmarks, MGSER-SAM has demonstrated a consistent ability to outperform existing baselines in all three CL scenarios. Comparing to the representative memory replay-based baselines ER and DER++, MGSER-SAM not only improves the testing accuracy by $24.4\%$ and $17.6\%$ respectively, but also achieves the lowest forgetting on each benchmark.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
