HPCR：面向在线持续学习的整体代理对比回放方法
HPCR: Holistic Proxy-based Contrastive Replay for Online Continual Learning
AI总结：
针对在线持续学习中的灾难性遗忘问题，本文提出HPCR方法，通过对比组件、温度组件和蒸馏组件分别改进PCR在特征提取、泛化和抗遗忘方面的不足，并在四个数据集上验证了其优越性。
摘要：
在线持续学习旨在开发一种神经网络，使其能够在单次遍历在线数据流的过程中持续学习新数据，但通常会遭受灾难性遗忘。现有的基于回放的方法通过以基于代理或基于对比的回放方式重放部分旧数据来缓解遗忘，但每种方式各有其不足。我们之前的工作提出了一种新颖的基于回放的方法，称为基于代理的对比回放（PCR），该方法通过实现两种回放方式的互补优势来解决这些不足。在本工作中，我们进一步对 PCR 进行了梯度和局限性分析。分析结果表明，PCR 在模型的特征提取、泛化能力和抗遗忘能力方面仍有进一步改进的空间。因此，我们开发了一种更先进的方法，命名为整体代理对比回放（HPCR）。HPCR 由三个组件组成，每个组件分别解决 PCR 的一个局限性。对比组件有条件地将锚点到样本对纳入 PCR，从而提高特征提取能力。第二个是温度组件，它根据梯度影响将温度系数解耦为两部分，并为它们设置不同的值以增强泛化能力。第三个是蒸馏组件，它通过额外的损失项约束学习过程，以提高抗遗忘能力。在四个数据集上的实验一致证明了 HPCR 优于各种最先进的方法。
英文摘要：
Online continual learning, aimed at developing a neural network that continuously learns new data from a single pass over an online data stream, generally suffers from catastrophic forgetting. Existing replay-based methods alleviate forgetting by replaying partial old data in a proxy-based or contrastive-based replay manner, each with its own shortcomings. Our previous work proposes a novel replay-based method called proxy-based contrastive replay (PCR), which handles the shortcomings by achieving complementary advantages of both replay manners. In this work, we further conduct gradient and limitation analysis of PCR. The analysis results show that PCR still can be further improved in feature extraction, generalization, and anti-forgetting capabilities of the model. Hence, we develop a more advanced method named holistic proxy-based contrastive replay (HPCR). HPCR consists of three components, each tackling one of the limitations of PCR. The contrastive component conditionally incorporates anchor-to-sample pairs to PCR, improving the feature extraction ability. The second is a temperature component that decouples the temperature coefficient into two parts based on their gradient impacts and sets different values for them to enhance the generalization ability. The third is a distillation component that constrains the learning process with additional loss terms to improve the anti-forgetting ability. Experiments on four datasets consistently demonstrate the superiority of HPCR over various state-of-the-art methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
