增强一致性与减轻偏差：一种用于增量学习的数据回放方法
Enhancing Consistency and Mitigating Bias: A Data Replay Approach for Incremental Learning
AI总结：
本文针对增量学习中的灾难性遗忘问题，提出CwD方法，通过KL散度损失减少反转数据与真实数据的不一致性，并引入正则化项平衡类别权重，在多个基准上取得持续性能提升。
摘要：
深度学习系统在从一系列任务中学习时容易发生灾难性遗忘，因为学习新任务时无法获得先前任务的旧数据。为解决这一问题，一些方法提出在新任务学习期间回放先前任务的数据，通常使用额外内存来存储回放数据。然而，由于内存限制和数据隐私问题，这种方法在实际中并不被期望采用。相反，无数据回放方法从分类模型中反转样本。尽管这些方法有效，但它们面临反转训练数据与真实训练数据之间的不一致性问题，而这一问题在近期工作中被忽视。为此，我们提出通过一些简化和假设来定量衡量数据一致性。利用这一度量，我们获得了洞察，从而开发了一种新的损失函数来减少不一致性。具体而言，该损失在联合多元高斯假设下最小化反转数据与真实数据分布之间的KL散度，这在持续学习中易于实现。此外，我们观察到旧类别的权重范数随着学习进展不断减小。我们分析了原因，并提出一个正则化项来平衡类别权重，使旧类别样本更具可区分性。总之，我们引入了用于类别增量学习的一致性增强数据回放与去偏分类器（CwD）。在CIFAR-100、Tiny-ImageNet和ImageNet100上的大量实验表明，与先前方法相比，CwD的性能持续提升。
英文摘要：
Deep learning systems are prone to catastrophic forgetting when learning from a sequence of tasks, as old data from previous tasks is unavailable when learning a new task. To address this, some methods propose replaying data from previous tasks during new task learning, typically using extra memory to store replay data. However, it is not expected in practice due to memory constraints and data privacy issues. Instead, data-free replay methods invert samples from the classification model. While effective, these methods face inconsistencies between inverted and real training data, overlooked in recent works. To that effect, we propose to measure the data consistency quantitatively by some simplification and assumptions. Using this measurement, we gain insight to develop a novel loss function that reduces inconsistency. Specifically, the loss minimizes the KL divergence between distributions of inverted and real data under a tied multivariate Gaussian assumption, which is simple to implement in continual learning. Additionally, we observe that old class weight norms decrease continually as learning progresses. We analyze the reasons and propose a regularization term to balance class weights, making old class samples more distinguishable. To conclude, we introduce Consistency-enhanced data replay with a Debiased classifier for class incremental learning (CwD). Extensive experiments on CIFAR-100, Tiny-ImageNet, and ImageNet100 show consistently improved performance of CwD compared to previous approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
