一种用于持续半监督学习的软最近邻框架
A soft nearest-neighbor framework for continual semi-supervised learning
AI总结：
针对持续半监督学习中未标注数据表示遗忘和标注样本过拟合问题，提出利用非参数最近邻分类器非线性划分特征空间，在低监督下实现SOTA并有效扩展至复杂数据集。
摘要：
尽管取得了重大进展，但最先进的持续学习方法的性能仍依赖于全标注数据这一不切实际的场景。在本文中，我们应对这一挑战，并提出了一种持续半监督学习方法——即并非所有数据样本都被标注的设定。该场景下的一个主要问题是模型会遗忘未标记数据的表示，并对已标记样本产生过拟合。我们利用最近邻分类器的能力，得益于其非参数特性，非线性地划分特征空间并灵活地对底层数据分布进行建模。这使模型能够为当前任务学习到强大的表示，并从先前任务中蒸馏出相关信息。我们进行了详尽的实验评估，结果表明我们的方法以巨大优势超越了所有现有方法，在持续半监督学习范式上确立了坚实的最先进水平。例如，在CIFAR-100上，即使我们使用的监督量至少少30倍（0.8%对比25%的标注），我们也超越了其他几种方法。最后，我们的方法在低分辨率和高分辨率图像上均表现良好，并能无缝扩展到更复杂的数据集，如ImageNet-100。代码已在https://github.com/kangzhiq/NNCSL上公开发布。
英文摘要：
Despite significant advances, the performance of state-of-the-art continual learning approaches hinges on the unrealistic scenario of fully labeled data. In this paper, we tackle this challenge and propose an approach for continual semi-supervised learning--a setting where not all the data samples are labeled. A primary issue in this scenario is the model forgetting representations of unlabeled data and overfitting the labeled samples. We leverage the power of nearest-neighbor classifiers to nonlinearly partition the feature space and flexibly model the underlying data distribution thanks to its non-parametric nature. This enables the model to learn a strong representation for the current task, and distill relevant information from previous tasks. We perform a thorough experimental evaluation and show that our method outperforms all the existing approaches by large margins, setting a solid state of the art on the continual semi-supervised learning paradigm. For example, on CIFAR-100 we surpass several others even when using at least 30 times less supervision (0.8% vs. 25% of annotations). Finally, our method works well on both low and high resolution images and scales seamlessly to more complex datasets such as ImageNet-100. The code is publicly available on https://github.com/kangzhiq/NNCSL
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
