OrCo：通过正交性与对比实现少样本类增量学习的更好泛化
OrCo: Towards Better Generalization via Orthogonality and Contrast for Few-Shot Class-Incremental Learning
AI总结：
针对少样本类增量学习中的灾难性遗忘与过拟合问题，提出OrCo框架，通过监督与自监督对比损失及OrCo损失实现特征正交性，最大化类间边界并预留空间，在三个基准数据集上达到SOTA。
摘要：
少样本类增量学习（FSCIL）引入了一种问题空间随有限数据扩展的范式。随着数据增量到达，FSCIL方法内在地面临灾难性遗忘的挑战，使模型容易覆盖先前获取的知识。此外，鉴于任何给定时刻可用的标注样本稀缺，模型可能容易过拟合，并难以在广泛的预训练与有限的增量数据之间取得平衡。为了应对这些挑战，我们提出了基于两个核心原则的OrCo框架：表示空间中特征的正交性，以及对比学习。具体而言，我们在预训练阶段采用监督和自监督对比损失相结合的方式，提升了嵌入空间的泛化能力。此外，我们引入OrCo损失以解决增量会话期间因数据限制而产生的挑战。通过特征空间扰动和类间正交性，OrCo损失最大化边界并为后续增量数据预留空间。这进而确保在特征空间中容纳新类别的同时不损害先前获取的知识。我们的实验结果展示了在三个基准数据集上的最先进性能，包括mini-ImageNet、CIFAR100和CUB数据集。代码可在https://github.com/noorahmedds/OrCo获取。
英文摘要：
Few-Shot Class-Incremental Learning (FSCIL) introduces a paradigm in which the problem space expands with limited data. FSCIL methods inherently face the challenge of catastrophic forgetting as data arrives incrementally, making models susceptible to overwriting previously acquired knowledge. Moreover, given the scarcity of labeled samples available at any given time, models may be prone to overfitting and find it challenging to strike a balance between extensive pretraining and the limited incremental data. To address these challenges, we propose the OrCo framework built on two core principles: features' orthogonality in the representation space, and contrastive learning. In particular, we improve the generalization of the embedding space by employing a combination of supervised and self-supervised contrastive losses during the pretraining phase. Additionally, we introduce OrCo loss to address challenges arising from data limitations during incremental sessions. Through feature space perturbations and orthogonality between classes, the OrCo loss maximizes margins and reserves space for the following incremental data. This, in turn, ensures the accommodation of incoming classes in the feature space without compromising previously acquired knowledge. Our experimental results showcase state-of-the-art performance across three benchmark datasets, including mini-ImageNet, CIFAR100, and CUB datasets. Code is available at https://github.com/noorahmedds/OrCo
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
