错误分类样本中的魔鬼：走向统一的开集识别
The Devil is in the Wrongly-classified Samples: Towards Unified Open-set Recognition
摘要：
开集识别（OSR）的目的是识别在训练过程中没有看到的测试样本。最近，统一开集识别（UOSR）已被提出，拒绝不仅未知的样本，但错误分类的样本，这往往是在现实世界中更实用。UOSR自提出以来很少引起关注，但我们发现有时它甚至比OSR更实用，因为已知但错误分类的样本的评估结果也像未知样本一样是错误的。在本文中，我们深入分析了UOSR任务在不同的培训和评估设置，揭示了这一有前途的研究方向。为了这个目的，我们首先评估UOSR性能的几种OSR方法，并显示一个显着的发现，UOSR性能始终超过OSR性能的同一方法的大幅度。我们发现，原因在于已知但错误分类的样本，因为它们的不确定性分布非常接近未知样本，而不是已知和正确分类的样本。其次，我们分析了OSR的两个训练设置（即，预培训和离群值暴露）影响UOSR。我们发现，虽然它们都有利于区分已知和正确分类的样本与未知样本，但预训练也有助于识别已知但错误分类的样本，而离群值暴露则不然。除了不同的训练设置，我们还为UOSR制定了一个新的评估设置，称为Few-Shot UOSR，在评估过程中每个未知类只有一个或五个样本，以帮助识别未知样本。我们建议FS-KNNS的Few-Shot UOSR实现国家的最先进的性能在所有设置。
英文摘要：
Open-set Recognition (OSR) aims to identify test samples whose classes are not seen during the training process. Recently, Unified Open-set Recognition (UOSR) has been proposed to reject not only unknown samples but also known but wrongly classified samples, which tends to be more practical in real-world applications. The UOSR draws little attention since it is proposed, but we find sometimes it is even more practical than OSR in the real world applications, as evaluation results of known but wrongly classified samples are also wrong like unknown samples. In this paper, we deeply analyze the UOSR task under different training and evaluation settings to shed light on this promising research direction. For this purpose, we first evaluate the UOSR performance of several OSR methods and show a significant finding that the UOSR performance consistently surpasses the OSR performance by a large margin for the same method. We show that the reason lies in the known but wrongly classified samples, as their uncertainty distribution is extremely close to unknown samples rather than known and correctly classified samples. Second, we analyze how the two training settings of OSR (i.e., pre-training and outlier exposure) influence the UOSR. We find although they are both beneficial for distinguishing known and correctly classified samples from unknown samples, pre-training is also helpful for identifying known but wrongly classified samples while outlier exposure is not. In addition to different training settings, we also formulate a new evaluation setting for UOSR which is called few-shot UOSR, where only one or five samples per unknown class are available during evaluation to help identify unknown samples. We propose FS-KNNS for the few-shot UOSR to achieve state-of-the-art performance under all settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
