arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2112.00337cs.CVcs.LG
收藏

深度神经网络未知检测能力的统一基准

A Unified Benchmark for the Unknown Detection Capability of Deep Neural Networks

Seoul National University of Science and Technology(首尔国立科技大学)

Jihyo Kim, Jiin Koo, Sangheum Hwang

AI总结:

研究指出误分类检测、开集识别和分布外检测本质相同,提出未知检测任务并构建统一基准,实验发现Deep Ensemble表现最优但所有方法仅对特定未知有效。

摘要:

深度神经网络在各种任务上取得了出色的性能,但存在一个关键问题:即使对完全未知的样本也会做出过度自信的预测。许多研究成功过滤了这些未知样本,但仅考虑了狭窄且特定的任务,如误分类检测、开集识别或分布外检测。在本研究中,我们认为这些任务应被视为本质上相同的问题,因为理想模型应具备对所有这些任务的检测能力。因此,我们引入了未知检测任务,整合了以往各个独立任务,以严格检验深度神经网络在广泛未知样本上的检测能力。为此,我们构建了不同规模的统一基准数据集,并比较了现有流行方法的未知检测能力。我们发现,Deep Ensemble 在检测未知样本方面始终优于其他方法;然而,所有方法仅对特定类型的未知有效。可复现代码和基准数据集可在 https://github.com/daintlab/unknown-detection-benchmarks 获取。

英文摘要:

Deep neural networks have achieved outstanding performance over various tasks, but they have a critical issue: over-confident predictions even for completely unknown samples. Many studies have been proposed to successfully filter out these unknown samples, but they only considered narrow and specific tasks, referred to as misclassification detection, open-set recognition, or out-of-distribution detection. In this work, we argue that these tasks should be treated as fundamentally an identical problem because an ideal model should possess detection capability for all those tasks. Therefore, we introduce the unknown detection task, an integration of previous individual tasks, for a rigorous examination of the detection capability of deep neural networks on a wide spectrum of unknown samples. To this end, unified benchmark datasets on different scales were constructed and the unknown detection capabilities of existing popular methods were subject to comparison. We found that Deep Ensemble consistently outperforms the other approaches in detecting unknowns; however, all methods are only successful for a specific type of unknown. The reproducible code and benchmark datasets are available at https://github.com/daintlab/unknown-detection-benchmarks .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑