通过选择性梯度投影缓解持续学习中的遗忘
Mitigating Forgetting in Continual Learning with Selective Gradient Projection
AI总结：
本文提出SFAO方法，通过余弦相似度和分层门控调节梯度方向，实现受控遗忘并平衡可塑性与稳定性，实验显示其在持续学习基准上表现优异，内存成本降低90%。
摘要：
随着神经网络在动态环境中的广泛应用，它们面临灾难性遗忘的挑战，即在适应新任务时倾向于覆盖先前学习的知识，导致早期任务性能严重下降。我们提出了选择性遗忘感知优化（SFAO），一种动态方法，通过余弦相似度和分层门控调节梯度方向，实现受控遗忘并平衡可塑性与稳定性。SFAO通过可调节机制选择性地投影、接受或丢弃更新，使用高效的蒙特卡洛近似。在标准持续学习基准上的实验表明，SFAO在准确性上具有竞争力，内存成本显著降低90%，并在MNIST数据集上改善了遗忘表现，使其适用于资源受限的场景。
英文摘要：
As neural networks are increasingly deployed in dynamic environments, they face the challenge of catastrophic forgetting, the tendency to overwrite previously learned knowledge when adapting to new tasks, resulting in severe performance degradation on earlier tasks. We propose Selective Forgetting-Aware Optimization (SFAO), a dynamic method that regulates gradient directions via cosine similarity and per-layer gating, enabling controlled forgetting while balancing plasticity and stability. SFAO selectively projects, accepts, or discards updates using a tunable mechanism with efficient Monte Carlo approximation. Experiments on standard continual learning benchmarks show that SFAO achieves competitive accuracy with markedly lower memory cost, a 90$\%$ reduction, and improved forgetting on MNIST datasets, making it suitable for resource-constrained scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
