ErrorEraser：通过遗忘数据偏差改进持续学习
ErrorEraser: Unlearning Data Bias for Improved Continual Learning
AI总结：
本文提出通用插件ErrorEraser，通过识别偏差样本并擦除错误记忆，在三类持续学习方法上提升准确率并降低遗忘率。
摘要：
持续学习（Continual Learning, CL）的主要目标是保留知识以防止灾难性遗忘，并迁移知识以促进新任务学习。与传统方法不同，我们提出一种新颖视角：CL不仅需要防止遗忘，还需要有意图的遗忘。这源于现有CL方法忽视了真实世界数据中的偏差，导致模型学习到虚假关联，并在任务之间迁移和放大。从特征提取和预测结果中，我们发现数据偏差同时降低了CL保留和迁移知识的能力。为解决该问题，我们提出ErrorEraser，这是一个通用插件，用于移除CL中由偏差导致的错误记忆，从而提升新任务和旧任务的性能。ErrorEraser包含两个模块：错误识别与错误擦除。前者在无需先验知识的情况下学习任务数据在特征空间中的概率密度分布，从而准确识别潜在偏差样本。后者通过移动代表性离群样本的决策空间，确保仅擦除错误知识。此外，我们设计了增量式特征分布学习策略，以降低下游任务错误识别期间的资源开销。大量实验结果表明，ErrorEraser显著缓解了数据偏差的负面影响，在三类CL方法上均实现了更高准确率和更低遗忘率。代码可在https://github.com/diadai/ErrorEraser获取。
英文摘要：
Continual Learning (CL) primarily aims to retain knowledge to prevent catastrophic forgetting and transfer knowledge to facilitate learning new tasks. Unlike traditional methods, we propose a novel perspective: CL not only needs to prevent forgetting, but also requires intentional forgetting.This arises from existing CL methods ignoring biases in real-world data, leading the model to learn spurious correlations that transfer and amplify across tasks. From feature extraction and prediction results, we find that data biases simultaneously reduce CL's ability to retain and transfer knowledge. To address this, we propose ErrorEraser, a universal plugin that removes erroneous memories caused by biases in CL, enhancing performance in both new and old tasks. ErrorEraser consists of two modules: Error Identification and Error Erasure. The former learns the probability density distribution of task data in the feature space without prior knowledge, enabling accurate identification of potentially biased samples. The latter ensures only erroneous knowledge is erased by shifting the decision space of representative outlier samples. Additionally, an incremental feature distribution learning strategy is designed to reduce the resource overhead during error identification in downstream tasks. Extensive experimental results show that ErrorEraser significantly mitigates the negative impact of data biases, achieving higher accuracy and lower forgetting rates across three types of CL methods. The code is available at https://github.com/diadai/ErrorEraser.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
