arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.09972cs.LGcs.CVcs.DS
收藏

邻域平均法改进离群点检测器

Neighborhood Averaging for Improving Outlier Detectors

Northwestern Polytechnical University(西北工业大学);Singapore Institute of Technology(新加坡理工大学);University of Eastern Finland(东芬兰大学)

Jiawei Yang, Susanto Rahardja, Pasi Franti

摘要:

我们假设相似的对象应该具有相似的离群值分数。据我们所知,所有现有的离群值检测器都独立地计算每个对象的离群值分数,而不管其他对象的离群值分数。因此,它们不能保证相似的对象具有相似的离群值分数。为了验证我们提出的假设,我们提出了一种离群值检测器的离群值得分后处理技术,称为邻域平均(NA),它关注对象及其邻居,并保证它们具有比原始得分更相似的离群值得分。给定一个对象及其来自任何离群值检测器的离群值得分,NA通过将其与其k个最近邻居的得分相结合来修改其离群值得分。我们证明了NA的有效性,通过使用著名的k-近邻(k-NN)。实验结果表明,NA将所有10个测试的基线检测器平均提高了13%(从0.70到0.79 AUC)。此外,即使是已经基于k-NN的离群检测器也得到了改进。实验还表明,在某些应用中,当检测器与NA联合使用时,检测器的选择不再重要,这可能对通常认为的数据模型是最重要因素的想法提出挑战。我们在www.outlierNet.com上打开代码以获得可重复性。

英文摘要:

We hypothesize that similar objects should have similar outlier scores. To our knowledge, all existing outlier detectors calculate the outlier score for each object independently regardless of the outlier scores of the other objects. Therefore, they do not guarantee that similar objects have similar outlier scores. To verify our proposed hypothesis, we propose an outlier score post-processing technique for outlier detectors, called neighborhood averaging(NA), which pays attention to objects and their neighbors and guarantees them to have more similar outlier scores than their original scores. Given an object and its outlier score from any outlier detector, NA modifies its outlier score by combining it with its k nearest neighbors' scores. We demonstrate the effectivity of NA by using the well-known k-nearest neighbors (k-NN). Experimental results show that NA improves all 10 tested baseline detectors by 13% (from 0.70 to 0.79 AUC) on average evaluated on nine real-world datasets. Moreover, even outlier detectors that are already based on k-NN are also improved. The experiments also show that in some applications, the choice of detector is no more significant when detectors are jointly used with NA, which may pose a challenge to the generally considered idea that the data model is the most important factor. We open our code on www.outlierNet.com for reproducibility.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑