基于平衡梯度样本检索的代理类持续学习知识保留增强方法
Balanced Gradient Sample Retrieval for Enhanced Knowledge Retention in Proxy-based Continual Learning
AI总结：
提出一种在监督对比学习框架下平衡利用梯度冲突与对齐样本的检索策略，通过增强参数空间对齐与样本多样性来缓解灾难性遗忘和代理漂移，在视觉持续学习基准上实现了最先进的抗遗忘性能。
摘要：
深度神经网络中的持续学习常常遭受灾难性遗忘，即在后续训练期间先前任务的表征会被覆盖。我们提出了一种从记忆缓冲区进行样本检索的新策略，该策略在监督对比学习框架内，同时利用梯度冲突样本和梯度对齐样本，以有效保留关于过去任务的知识。选择梯度冲突样本是因为它们有潜力通过重新对齐梯度来减少干扰，从而保留过去任务的知识。同时，引入梯度对齐样本是为了强化跨任务的稳定且共享的表征。通过平衡来自冲突样本的梯度校正与来自对齐样本的对齐强化，我们的方法提升了检索实例间的多样性，并在参数空间中实现了更优的对齐，显著增强了知识保留并缓解了代理漂移。实证结果表明，同时使用两种样本类型的方法优于仅依赖单一类型或随机检索的方法。在计算机视觉领域流行的持续学习基准上的实验验证了我们方法在缓解遗忘方面的最先进性能，同时在新任务上保持了具竞争力的准确率。
英文摘要：
Continual learning in deep neural networks often suffers from catastrophic forgetting, where representations for previous tasks are overwritten during subsequent training. We propose a novel sample retrieval strategy from the memory buffer that leverages both gradient-conflicting and gradient-aligned samples to effectively retain knowledge about past tasks within a supervised contrastive learning framework. Gradient-conflicting samples are selected for their potential to reduce interference by re-aligning gradients, thereby preserving past task knowledge. Meanwhile, gradient-aligned samples are incorporated to reinforce stable, shared representations across tasks. By balancing gradient correction from conflicting samples with alignment reinforcement from aligned ones, our approach increases the diversity among retrieved instances and achieves superior alignment in parameter space, significantly enhancing knowledge retention and mitigating proxy drift. Empirical results demonstrate that using both sample types outperforms methods relying solely on one sample type or random retrieval. Experiments on popular continual learning benchmarks in computer vision validate our method's state-of-the-art performance in mitigating forgetting while maintaining competitive accuracy on new tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
