UER：一种用于在线持续学习的启发式偏差处理方法
UER: A Heuristic Bias Addressing Approach for Online Continual Learning
AI总结：
针对在线持续学习中的偏差问题，提出启发式方法UER，通过分解点积logits利用范数因子平衡新旧知识，实现优于现有方法的性能。
摘要：
在线持续学习旨在通过单次遍历数据，从连续数据流中持续训练神经网络。作为最有效的方法，基于排练的方法重放部分先前数据。现有方法中常用的预测器倾向于生成偏向当前数据类别的点积logits，这被称为偏差问题和遗忘现象。已有许多方法被提出以通过纠正偏差来克服遗忘问题；然而，它们在在线模式下仍需改进。在本文中，我们尝试通过一种更直接、更高效的方法来解决偏差问题。通过将点积logits分解为角度因子和范数因子，我们经验性地发现偏差问题主要出现在角度因子中，该因子可作为cosine logits用于学习新知识。相反，被现有方法遗弃的范数因子有助于记住历史知识。基于这一观察，我们直观地提出利用范数因子来平衡新旧知识以解决偏差。为此，我们开发了一种名为无偏经验回放（UER）的启发式方法。UER仅通过角度因子学习当前样本，并进一步通过范数和角度因子重放先前样本。在三个数据集上的大量实验表明，UER实现了优于各种最先进方法的性能。代码在 https://github.com/FelixHuiweiLin/UER。
英文摘要：
Online continual learning aims to continuously train neural networks from a continuous data stream with a single pass-through data. As the most effective approach, the rehearsal-based methods replay part of previous data. Commonly used predictors in existing methods tend to generate biased dot-product logits that prefer to the classes of current data, which is known as a bias issue and a phenomenon of forgetting. Many approaches have been proposed to overcome the forgetting problem by correcting the bias; however, they still need to be improved in online fashion. In this paper, we try to address the bias issue by a more straightforward and more efficient method. By decomposing the dot-product logits into an angle factor and a norm factor, we empirically find that the bias problem mainly occurs in the angle factor, which can be used to learn novel knowledge as cosine logits. On the contrary, the norm factor abandoned by existing methods helps remember historical knowledge. Based on this observation, we intuitively propose to leverage the norm factor to balance the new and old knowledge for addressing the bias. To this end, we develop a heuristic approach called unbias experience replay (UER). UER learns current samples only by the angle factor and further replays previous samples by both the norm and angle factors. Extensive experiments on three datasets show that UER achieves superior performance over various state-of-the-art methods. The code is in https://github.com/FelixHuiweiLin/UER.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
