MANGO：面向在线持续学习的元适应网络梯度优化
MANGO: Meta-Adaptive Network Gradient Optimization for Online Continual Learning
AI总结：
本文提出MANGO框架，通过梯度门控和元学习正则化平衡持续学习中的稳定性与可塑性，实现对过去任务遗忘的克服和新任务高效学习。
摘要：
在在线持续学习（OCL）中，神经网络在单次通过中从非平稳数据流中依次学习，仅能访问有限的内存回放缓冲区。这与离线持续学习形成鲜明对比，后者依赖多个epoch训练大型数据集。OCL的主要挑战是克服对过去任务的灾难性遗忘（稳定性）的同时高效学习新任务（可塑性）。现有方法通过回放式复习、输出级蒸馏、固定正则化或当前数据上的元学习来对抗遗忘。然而，这些方法存在局限：复习引入存储样本偏差；蒸馏在输出分布上操作而无法调节参数更新；固定正则化对参数施加惩罚而不考虑敏感性；仅基于数据流的元学习缺乏反馈控制的参数更新。我们提出元适应网络梯度优化（MANGO），一种OCL框架，通过梯度门控和元学习正则化平衡稳定性与可塑性。梯度门控根据敏感性调整参数更新，防止破坏性更新。元学习正则化适应稳定性系数，评估参数更新对回放的影响。在MANGO中，回放同时充当训练信号和遗忘评估器。我们在三个标准OCL基准数据集上评估了我们的方法。MANGO在多个基准上优于强基线方法，取得最先进的结果，并在不同回放大小下保持一致性能。在CLEAR-10上的领域增量学习和CIFAR-100和Tiny-ImageNet上的类别增量学习中，它在所有基线中取得最高准确率，并实现正向反馈转移，克服CLEAR-10上的遗忘。
英文摘要：
In Online Continual Learning (OCL), a neural network sequentially learns from a non-stationary data stream in a single-pass with access only to a limited memory replay buffer. This contrasts sharply with off-line continual learning where training is multiple epoch dependent on large datasets. The main challenge faced by OCL is to overcome catastrophic forgetting of past tasks (stability) while learning new ones efficiently (plasticity). Existing methods counter forgetting via replay-based rehearsal, output level distillation, fixed regularization, or meta-learning on the current data. However, these methods have limitations: rehearsal introduces a stored sample bias; distillation operates on output-distributions without modulating parameter updates; fixed-regularization penalizes parameters irrespective of sensitivity; stream-only meta-learning lacks a feedback controlled parameter update. We propose Meta-Adaptive Network Gradient Optimization (MANGO), an OCL framework that balances stability-plasticity via gradient-gating and meta-learned regularization. Gradient-gating scales parameter updates based on sensitivity, preventing destructive updates. Meta-learned regularization adapts stability coefficients, evaluating the effect of parameter update on replay. In MANGO, replay acts as both a training signal and a forgetting evaluator. We evaluated our method on three standard OCL benchmark datasets. MANGO outperforms strong baselines, achieving state-of-the-art results with consistent performance across replay sizes. In domain incremental learning on CLEAR-10 and class incremental learning on CIFAR-100 and Tiny-ImageNet, it achieves highest accuracy among all baselines and achieves positive Backward Transfer, overcoming forgetting on CLEAR-10.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
