平衡类增量学习中的因果效应
Balancing the Causal Effects in Class-Incremental Learning
AI总结：
针对类增量学习中新旧数据因果效应冲突导致灾难性遗忘的问题，提出BaCE方法构建双向因果路径平衡因果效应，在多任务上显著优于现有方法。
摘要：
类增量学习（Class-Incremental Learning，CIL）是实现通用人工智能的一个实际且具挑战性的问题。最近，预训练模型（Pre-Trained Models，PTMs）在视觉和自然语言处理任务中均取得了突破。尽管最近的研究表明PTMs具有顺序学习的潜在能力，但大量工作表明缓解PTMs的灾难性遗忘是必要的。通过对CIL的初步研究和因果分析，我们揭示了问题的症结在于新数据和旧数据之间的因果效应不平衡。具体来说，新数据鼓励模型适应新类，同时阻碍旧类的适应。类似地，旧数据鼓励模型适应旧类，同时阻碍新类的适应。换言之，从因果角度来看，新类和旧类之间的适应过程存在冲突。为了缓解这个问题，我们提出在CIL中平衡因果效应（Balancing the Causal Effects，BaCE）。具体而言，BaCE提出了两个目标，分别用于构建从新数据和旧数据到新类和旧类预测的因果路径。通过这种方式，模型被鼓励利用来自新数据和旧数据的因果效应来适应所有类，从而缓解因果不平衡问题。我们在持续图像分类、持续文本分类和持续命名实体识别上进行了大量实验。实证结果表明，BaCE在不同任务和设置上均优于一系列CIL方法。
英文摘要：
Class-Incremental Learning (CIL) is a practical and challenging problem for achieving general artificial intelligence. Recently, Pre-Trained Models (PTMs) have led to breakthroughs in both visual and natural language processing tasks. Despite recent studies showing PTMs' potential ability to learn sequentially, a plethora of work indicates the necessity of alleviating the catastrophic forgetting of PTMs. Through a pilot study and a causal analysis of CIL, we reveal that the crux lies in the imbalanced causal effects between new and old data. Specifically, the new data encourage models to adapt to new classes while hindering the adaptation of old classes. Similarly, the old data encourages models to adapt to old classes while hindering the adaptation of new classes. In other words, the adaptation process between new and old classes conflicts from the causal perspective. To alleviate this problem, we propose Balancing the Causal Effects (BaCE) in CIL. Concretely, BaCE proposes two objectives for building causal paths from both new and old data to the prediction of new and classes, respectively. In this way, the model is encouraged to adapt to all classes with causal effects from both new and old data and thus alleviates the causal imbalance problem. We conduct extensive experiments on continual image classification, continual text classification, and continual named entity recognition. Empirical results show that BaCE outperforms a series of CIL methods on different tasks and settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
