arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 0903.3257cs.LGcs.IR
收藏

一种新的基于局部距离的离散真实数据离群点检测方法

A New Local Distance-Based Outlier Detection Approach for Scattered Real-World Data

Australian National University(澳大利亚国立大学);National ICT Australia (NICTA)(澳大利亚国家信息与通信技术研究院(NICTA));CSIRO Mathematical and Information Sciences(联邦科学与工业研究组织数学与信息科学研究所)

Ke Zhang, Marcus Hutter, Huidong Jin

摘要:

检测与其余数据集完全不同或不一致的离群值是现实世界KDD应用中的一个主要挑战。现有的离群点检测方法由于隐含的数据模式和参数设置问题,在分散的真实世界数据集上是无效的。我们定义了一种新的“基于局部距离的离群因子”(LDOF)来度量分散数据集中对象的离群性,从而解决了这些问题。LDOF使用对象与其邻居的相对位置来确定对象偏离其邻居的程度。从理论上分析了LDOF的性质,包括LDOF的下界和误检概率,以及参数设置。为了方便在现实世界中的应用程序中的参数设置,我们采用了top-n技术在我们的离群值检测方法,其中只有最高LDOF值的对象被视为离群值。与传统的方法(如top-nKNN和top-nLOF)相比,我们的方法top-nLDOF在检测散乱数据中的离群点时更有效。它也更容易设置参数,因为它的性能在很大的参数值范围内相对稳定,正如真实世界和合成数据集上的实验结果所示。

英文摘要:

Detecting outliers which are grossly different from or inconsistent with the remaining dataset is a major challenge in real-world KDD applications. Existing outlier detection methods are ineffective on scattered real-world datasets due to implicit data patterns and parameter setting issues. We define a novel "Local Distance-based Outlier Factor" (LDOF) to measure the {outlier-ness} of objects in scattered datasets which addresses these issues. LDOF uses the relative location of an object to its neighbours to determine the degree to which the object deviates from its neighbourhood. Properties of LDOF are theoretically analysed including LDOF's lower bound and its false-detection probability, as well as parameter settings. In order to facilitate parameter settings in real-world applications, we employ a top-n technique in our outlier detection approach, where only the objects with the highest LDOF values are regarded as outliers. Compared to conventional approaches (such as top-n KNN and top-n LOF), our method top-n LDOF is more effective at detecting outliers in scattered data. It is also easier to set parameters, since its performance is relatively stable over a large range of parameter values, as illustrated by experimental results on both real-world and synthetic datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑