arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2103.12344cs.LGcs.CV
收藏

跨表示空间的联合分布用于分布外检测

Joint Distribution across Representation Space for Out-of-Distribution Detection

State Key Lab of Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室)

JingWei Xu, Siyuan Zhu, Zenan Li, Chang Xu

摘要:

深度神经网络(DNN)已经成为许多现代软件应用的关键部分。经过训练和验证后,DNN被部署为不可撤销的组件,并应用于实际场景。尽管大多数DNN都是用大量的训练数据精心构建的,但真实世界中的数据仍然不为DNN模型所知,这导致了运行时分布外(OOD)检测的关键要求。然而,许多现有的方法1)需要OOD数据用于分类器训练或参数调整,或者2)简单地将每个隐藏层的分数组合为用于OOD检测的特征的集合。在本文中,我们提出了一种新的观点,在分布数据的生成方式,它把他们的潜在功能产生的每个隐藏层作为一个联合分布跨表示空间。由于只有分布内的潜在特征在表示空间中被全面理解,因此分布内数据和面向对象数据之间的内在差异可以在没有任何面向对象数据干预的情况下自然地被揭示。具体来说,我们构建了一个生成模型,称为潜在序列高斯混合(LSGM),以描述分布内潜在特征是如何根据DNN在表示空间中的推理轨迹生成的。我们首先构造高斯混合模型(GMM)的基础上,分布潜在的功能,为每个隐藏层,然后连接GMM通过转移概率的推理痕迹。通过对主流OOD数据集和模型的实验评估,验证了该方法在OOD检测中的优越性。

英文摘要:

Deep neural networks (DNNs) have become a key part of many modern software applications. After training and validating, the DNN is deployed as an irrevocable component and applied in real-world scenarios. Although most DNNs are built meticulously with huge volumes of training data, data in the real world still remain unknown to the DNN model, which leads to the crucial requirement of runtime out-of-distribution (OOD) detection. However, many existing approaches 1) need OOD data for classifier training or parameter tuning, or 2) simply combine the scores of each hidden layer as an ensemble of features for OOD detection. In this paper, we present a novel outlook on in-distribution data in a generative manner, which takes their latent features generated from each hidden layer as a joint distribution across representation spaces. Since only the in-distribution latent features are comprehensively understood in representation space, the internal difference between in-distribution and OOD data can be naturally revealed without the intervention of any OOD data. Specifically, We construct a generative model, called Latent Sequential Gaussian Mixture (LSGM), to depict how the in-distribution latent features are generated in terms of the trace of DNN inference across representation spaces. We first construct the Gaussian Mixture Model (GMM) based on in-distribution latent features for each hidden layer, and then connect GMMs via the transition probabilities of the inference traces. Experimental evaluations on popular benchmark OOD datasets and models validate the superiority of the proposed method over the state-of-the-art methods in OOD detection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑