IMEX-Reg：用于持续学习的函数空间隐式-显式正则化
IMEX-Reg: Implicit-Explicit Regularization in the Function Space for Continual Learning
AI总结：
针对持续学习低缓冲区机制下的过拟合与先验信息丢失问题，提出IMEX-Reg方法，通过结合对比表示学习与一致性正则化进行隐式-显式正则化，显著提升了模型的泛化性能与鲁棒性。
摘要：
由于对先前获取知识的灾难性遗忘，持续学习（CL）仍然是深度神经网络面临的长期挑战之一。尽管基于排练的方法在缓解灾难性遗忘方面相当成功，但它们在缓冲样本上存在过拟合以及先验信息丢失的问题，阻碍了在低缓冲区机制下的泛化。受人类利用强归纳偏置进行学习的启发，我们提出了IMEX-Reg，以改善低缓冲区机制下CL中经验排练的泛化性能。具体而言，我们采用一种双管齐下的隐式-显式正则化方法，使用对比表示学习（CRL）和一致性正则化。为了进一步利用通过CRL学习到的表示之间的全局关系，我们提出了一种正则化策略，以引导分类器趋向于CRL单位超球面中的激活相关性。我们的结果表明，IMEX-Reg显著提高了泛化性能，并在几种CL场景中优于基于排练的方法。它对自然和对抗性损坏具有鲁棒性，且任务近因偏差更小。此外，我们提供了理论见解以进一步支持我们的设计决策。
英文摘要：
Continual learning (CL) remains one of the long-standing challenges for deep neural networks due to catastrophic forgetting of previously acquired knowledge. Although rehearsal-based approaches have been fairly successful in mitigating catastrophic forgetting, they suffer from overfitting on buffered samples and prior information loss, hindering generalization under low-buffer regimes. Inspired by how humans learn using strong inductive biases, we propose IMEX-Reg to improve the generalization performance of experience rehearsal in CL under low buffer regimes. Specifically, we employ a two-pronged implicit-explicit regularization approach using contrastive representation learning (CRL) and consistency regularization. To further leverage the global relationship between representations learned using CRL, we propose a regularization strategy to guide the classifier toward the activation correlations in the unit hypersphere of the CRL. Our results show that IMEX-Reg significantly improves generalization performance and outperforms rehearsal-based approaches in several CL scenarios. It is also robust to natural and adversarial corruptions with less task-recency bias. Additionally, we provide theoretical insights to support our design decisions further.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
