房间里吵闹的大象:你的分布外检测器对标签噪声鲁棒吗?
A noisy elephant in the room: Is your out-of-distribution detector robust to label noise?
AI总结:
本文系统评估了20种最先进OOD检测方法在训练标签含噪声场景下的性能,通过多数据集、多噪声类型与水平、多架构及检查点策略实验,揭示了错误分类的ID样本与OOD样本分离度差是现有方法被忽视的关键局限。
摘要:
检测不熟悉或意外的图像对于计算机视觉系统的安全部署至关重要。在分类任务中,检测模型训练域之外图像的任务被称为分布外(OOD)检测。尽管人们对开发事后OOD检测方法的研究兴趣日益增长,但关于当底层分类器不是在干净、精心整理的数据集上训练时这些方法表现如何的讨论相对较少。在这项工作中,我们在一个(更现实的)场景中仔细考察了20种最先进的OOD检测方法,该场景中用于训练底层分类器的标签不可靠(例如众包或网络抓取的标签)。跨不同数据集、噪声类型与水平、架构和检查点策略的大量实验深入揭示了类别标签噪声对OOD检测的影响,并表明错误分类的分布内(ID)样本与OOD样本之间分离度差是现有方法一个被忽视但重要的局限。代码:https://github.com/glhr/ood-labelnoise
英文摘要:
The ability to detect unfamiliar or unexpected images is essential for safe deployment of computer vision systems. In the context of classification, the task of detecting images outside of a model's training domain is known as out-of-distribution (OOD) detection. While there has been a growing research interest in developing post-hoc OOD detection methods, there has been comparably little discussion around how these methods perform when the underlying classifier is not trained on a clean, carefully curated dataset. In this work, we take a closer look at 20 state-of-the-art OOD detection methods in the (more realistic) scenario where the labels used to train the underlying classifier are unreliable (e.g. crowd-sourced or web-scraped labels). Extensive experiments across different datasets, noise types & levels, architectures and checkpointing strategies provide insights into the effect of class label noise on OOD detection, and show that poor separation between incorrectly classified ID samples vs. OOD samples is an overlooked yet important limitation of existing methods. Code: https://github.com/glhr/ood-labelnoise
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。