原型-样本关系蒸馏：迈向无回放持续学习
Prototype-Sample Relation Distillation: Towards Replay-Free Continual Learning
AI总结：
针对持续学习中旧任务数据不可用的问题，提出原型-样本关系蒸馏，在监督对比嵌入空间中持续演化类别原型并保持其相对相似性，实现无回放的任务与类别增量学习。
摘要：
在持续学习（CL）中，平衡有效适应与对抗灾难性遗忘是一项核心挑战。近期许多表现最佳的方法利用各种形式的先前任务数据（例如回放缓冲区）来解决灾难性遗忘问题。在许多现实场景中，能够访问先前任务数据可能受到限制，例如当任务数据敏感或专有时。为了摆脱使用先前任务数据的必要性，在本工作中，我们从已被证明较不易发生遗忘的强表示学习方法出发。我们提出一种整体方法，用于联合学习表示和类别原型，同时保持旧类别原型的相关性及其嵌入相似性。具体而言，样本被映射到一个嵌入空间，在该空间中使用监督对比损失学习表示。类别原型在同一潜在空间中持续演化，从而能够在任意时刻进行学习和预测。为了在不保留任何先前任务数据的情况下持续调整原型，我们提出一种新颖的蒸馏损失，它约束类别原型相对于新任务数据保持相对相似性。该方法在任务增量设置中取得了最先进的性能，优于依赖大量数据的方法，并在类别增量设置中提供了强劲性能，且没有使用任何存储的数据点。
英文摘要：
In Continual learning (CL) balancing effective adaptation while combating catastrophic forgetting is a central challenge. Many of the recent best-performing methods utilize various forms of prior task data, e.g. a replay buffer, to tackle the catastrophic forgetting problem. Having access to previous task data can be restrictive in many real-world scenarios, for example when task data is sensitive or proprietary. To overcome the necessity of using previous tasks' data, in this work, we start with strong representation learning methods that have been shown to be less prone to forgetting. We propose a holistic approach to jointly learn the representation and class prototypes while maintaining the relevance of old class prototypes and their embedded similarities. Specifically, samples are mapped to an embedding space where the representations are learned using a supervised contrastive loss. Class prototypes are evolved continually in the same latent space, enabling learning and prediction at any point. To continually adapt the prototypes without keeping any prior task data, we propose a novel distillation loss that constrains class prototypes to maintain relative similarities as compared to new task data. This method yields state-of-the-art performance in the task-incremental setting, outperforming methods relying on large amounts of data, and provides strong performance in the class-incremental setting without using any stored data points.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
