通过反向特征投影在持续学习中保持线性可分性
Preserving Linear Separability in Continual Learning by Backward Feature Projection
AI总结：
针对持续学习中的灾难性遗忘问题，提出反向特征投影（BFP）方法，允许新特征变化为旧特征的可学习线性变换，在保留旧类线性可分性的同时容纳新类，有效平衡稳定性与可塑性并提升模型性能。
摘要：
灾难性遗忘一直是持续学习中的主要挑战，在此场景下，模型需要学习新任务，同时对先前已见任务数据的访问受限或无法访问。为了应对这一挑战，人们提出了基于特征空间知识蒸馏的方法，并证明其能减少遗忘。然而，大多数特征蒸馏方法直接约束新特征去匹配旧特征，忽略了可塑性的需求。为了实现更好的稳定性与可塑性权衡，我们提出了反向特征投影（BFP），这是一种持续学习方法，允许新特征改变至旧特征的可学习线性变换的程度。BFP 保留了旧类的线性可分性，同时允许新特征方向的出现以容纳新类。BFP 可以与现有的经验回放方法集成，并大幅提升性能。我们还证明 BFP 有助于学习更好的表示空间，在该空间中，线性可分性在持续学习期间得到了很好的保留，并且线性探测实现了高分类准确率。代码可在 https://github.com/rvl-lab-utoronto/BFP 找到。
英文摘要：
Catastrophic forgetting has been a major challenge in continual learning, where the model needs to learn new tasks with limited or no access to data from previously seen tasks. To tackle this challenge, methods based on knowledge distillation in feature space have been proposed and shown to reduce forgetting. However, most feature distillation methods directly constrain the new features to match the old ones, overlooking the need for plasticity. To achieve a better stability-plasticity trade-off, we propose Backward Feature Projection (BFP), a method for continual learning that allows the new features to change up to a learnable linear transformation of the old features. BFP preserves the linear separability of the old classes while allowing the emergence of new feature directions to accommodate new classes. BFP can be integrated with existing experience replay methods and boost performance by a significant margin. We also demonstrate that BFP helps learn a better representation space, in which linear separability is well preserved during continual learning and linear probing achieves high classification accuracy. The code can be found at https://github.com/rvl-lab-utoronto/BFP
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
