arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2110.09276cs.LG
收藏

基于自然属性的偏移检测

Natural Attribute-based Shift Detection

KAIST(韩国科学技术院);University of Wisconsin-Madison(威斯康星大学麦迪逊分校)

Jeonghoon Park, Jimin Hong, Radhika Dua, Daehoon Gwak, Yixuan Li, Jaegul Choo, Edward Choi

摘要:

尽管深度网络在视觉、语言和医疗保健方面的表现令人印象深刻,但来自与训练分布不同的分布的样本的不可预测行为会导致部署中的严重问题。为了提高基于神经网络的分类器的可靠性,我们定义了一个新的任务,自然属性为基础的转变(NAS)检测,检测从训练分布的一些自然属性,如对象的年龄或图像的亮度移动的样本。使用现有数据集中存在的自然属性,我们引入了视觉,语言和医疗方面的基准数据集用于NAS检测。此外,我们对NAS数据集上的先验代表性分布外(OOD)检测方法进行了广泛的评估,并观察到其性能的不一致性。为了理解这一点,我们提供了一个NAS样本在特征空间中的位置和基于距离和置信度的OOD检测方法的性能之间的关系的分析。基于分析,我们将NAS样本分为三类,并进一步建议对训练目标进行简单修改,以获得能够检测所有NAS类别样本的改进OOD检测方法。

英文摘要:

Despite the impressive performance of deep networks in vision, language, and healthcare, unpredictable behaviors on samples from the distribution different than the training distribution cause severe problems in deployment. For better reliability of neural-network-based classifiers, we define a new task, natural attribute-based shift (NAS) detection, to detect the samples shifted from the training distribution by some natural attribute such as age of subjects or brightness of images. Using the natural attributes present in existing datasets, we introduce benchmark datasets in vision, language, and medical for NAS detection. Further, we conduct an extensive evaluation of prior representative out-of-distribution (OOD) detection methods on NAS datasets and observe an inconsistency in their performance. To understand this, we provide an analysis on the relationship between the location of NAS samples in the feature space and the performance of distance- and confidence-based OOD detection methods. Based on the analysis, we split NAS samples into three categories and further suggest a simple modification to the training objective to obtain an improved OOD detection method that is capable of detecting samples from all NAS categories.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑