用于持续自监督学习的伪负样本正则化
Regularizing with Pseudo-Negatives for Continual Self-Supervised Learning
AI总结：
本文提出伪负样本正则化框架PNR，利用当前与先前模型生成的伪负样本约束持续自监督学习，兼顾可塑性与稳定性，在CSSL表示学习中取得最先进性能。
摘要：
我们提出了一种新颖的伪负样本正则化（PNR）框架，用于有效的持续自监督学习（CSSL）。我们的PNR利用通过基于模型的增强获得的伪负样本，使得新学习的表示不会与过去已学到的内容相矛盾。具体而言，对于基于InfoNCE的对比学习方法，我们定义了从当前模型和先前模型获得的对称伪负样本，并将它们同时用于主损失项和正则化损失项。此外，我们将这一思想扩展到本质上不依赖负样本的非对比学习方法。对于这些方法，伪负样本被定义为先前模型对锚样本的不同增强版本产生的输出，并以非对称方式应用于正则化项。大量实验结果表明，我们的PNR框架通过有效平衡可塑性与稳定性之间的权衡，在CSSL过程中实现了最先进的表示学习性能。
英文摘要：
We introduce a novel Pseudo-Negative Regularization (PNR) framework for effective continual self-supervised learning (CSSL). Our PNR leverages pseudo-negatives obtained through model-based augmentation in a way that newly learned representations may not contradict what has been learned in the past. Specifically, for the InfoNCE-based contrastive learning methods, we define symmetric pseudo-negatives obtained from current and previous models and use them in both main and regularization loss terms. Furthermore, we extend this idea to non-contrastive learning methods which do not inherently rely on negatives. For these methods, a pseudo-negative is defined as the output from the previous model for a differently augmented version of the anchor sample and is asymmetrically applied to the regularization term. Extensive experimental results demonstrate that our PNR framework achieves state-of-the-art performance in representation learning during CSSL by effectively balancing the trade-off between plasticity and stability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
