面向冷启动无样本增量学习的弹性特征巩固
Elastic Feature Consolidation for Cold Start Exemplar-Free Incremental Learning
AI总结：
针对无样本类增量学习在冷启动场景下的特征漂移问题，提出弹性特征巩固方法，利用经验特征矩阵正则化漂移并更新高斯原型，有效平衡新任务学习与旧知识保持，显著优于现有方法。
摘要：
无样本类增量学习（EFCIL）旨在从一系列任务中学习，而无需访问先前任务的数据。在本文中，我们考虑了具有挑战性的冷启动场景，即第一个任务中没有足够的数据来学习高质量的主干网络。这对EFCIL来说尤其具有挑战性，因为它需要高可塑性，从而导致特征漂移，而这种漂移在无样本设置中很难补偿。为了解决这个问题，我们提出了一种简单有效的方法，通过在高度相关于先前任务的方向上正则化漂移来巩固特征表示，并利用原型来减少任务近因偏差。我们的方法称为弹性特征巩固（EFC），利用基于经验特征矩阵（EFM）的易于处理的二阶近似来估计特征漂移。EFM在特征空间中诱导出一个伪度量，我们用它来正则化重要方向上的特征漂移，并更新用于一种新型非对称交叉熵损失的高斯原型，该损失有效地平衡了原型排练与新任务的数据。在CIFAR-100、Tiny-ImageNet、ImageNet-Subset和ImageNet-1K上的实验结果表明，弹性特征巩固能够通过保持模型可塑性更好地学习新任务，并显著优于现有技术。
英文摘要：
Exemplar-Free Class Incremental Learning (EFCIL) aims to learn from a sequence of tasks without having access to previous task data. In this paper, we consider the challenging Cold Start scenario in which insufficient data is available in the first task to learn a high-quality backbone. This is especially challenging for EFCIL since it requires high plasticity, which results in feature drift which is difficult to compensate for in the exemplar-free setting. To address this problem, we propose a simple and effective approach that consolidates feature representations by regularizing drift in directions highly relevant to previous tasks and employs prototypes to reduce task-recency bias. Our method, called Elastic Feature Consolidation (EFC), exploits a tractable second-order approximation of feature drift based on an Empirical Feature Matrix (EFM). The EFM induces a pseudo-metric in feature space which we use to regularize feature drift in important directions and to update Gaussian prototypes used in a novel asymmetric cross entropy loss which effectively balances prototype rehearsal with data from new tasks. Experimental results on CIFAR-100, Tiny-ImageNet, ImageNet-Subset and ImageNet-1K demonstrate that Elastic Feature Consolidation is better able to learn new tasks by maintaining model plasticity and significantly outperform the state-of-the-art.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
