arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.10474cs.LGcs.CV
收藏

面向分布外检测的真实基准构建

Toward a Realistic Benchmark for Out-of-Distribution Detection

Politecnico di Torino(都灵理工大学)

Pietro Recalcati, Fabio Garcea, Luca Piano, Fabrizio Lamberti, Lia Morra

AI总结:

针对现有OOD检测基准多基于远分布外样本、缺乏真实场景复杂度的问题,研究提出基于ImageNet和Places365、按语义相似度划分分布内外类别的综合基准,实验表明检测效能受基准选择影响,置信度方法在近OOD样本上表现更优。

摘要:

深度神经网络已被越来越广泛地应用于各类技术与服务中,但仍极易受到分布外(OOD,即采样自与原始训练集不同分布的样本)的影响。解决这一问题的常用方法是赋予深度神经网络检测OOD样本的能力。目前已有多个用于设计和验证OOD检测技术的基准被提出,但其中很多都基于差异极大的远分布外样本,因此缺乏捕捉真实场景细微差异所需的复杂度。在本研究中,我们提出了一个基于ImageNet和Places365的综合性OOD检测基准,该基准根据类别与训练集的语义相似度,将各个类别划分为分布内或分布外。我们可采用多种技术确定哪些类别应被视为分布内,从而生成具有不同属性的基准。针对不同OOD检测技术的实验结果表明,其测得的效能取决于所选基准,且在近分布外样本检测上,基于置信度的技术表现可能优于基于分类器的技术。

英文摘要:

Deep neural networks are increasingly used in a wide range of technologies and services, but remain highly susceptible to out-of-distribution (OOD) samples, that is, drawn from a different distribution than the original training set. A common approach to address this issue is to endow deep neural networks with the ability to detect OOD samples. Several benchmarks have been proposed to design and validate OOD detection techniques. However, many of them are based on far-OOD samples drawn from very different distributions, and thus lack the complexity needed to capture the nuances of real-world scenarios. In this work, we introduce a comprehensive benchmark for OOD detection, based on ImageNet and Places365, that assigns individual classes as in-distribution or out-of-distribution depending on the semantic similarity with the training set. Several techniques can be used to determine which classes should be considered in-distribution, yielding benchmarks with varying properties. Experimental results on different OOD detection techniques show how their measured efficacy depends on the selected benchmark and how confidence-based techniques may outperform classifier-based ones on near-OOD samples.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑