arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1908.04000stat.MLcs.LGstat.AP
收藏

高维数据的异常检测

Anomaly Detection in High Dimensional Data

Monash University(莫纳什大学);University of Melbourne(墨尔本大学)

Priyanga Dilini Talagala, Rob J. Hyndman, Kate Smith-Miles

摘要:

HDoutliers算法是一种用于高维数据异常检测的强大无监督算法,具有很强的理论基础。然而,在某些情况下,它受到一些限制,严重阻碍了其性能水平。在这篇文章中,我们提出了一个算法,解决这些限制。我们将异常定义为明显偏离大多数的观察结果,具有较大的距离差距。利用极值理论计算异常阈值。使用各种合成和真实的数据集,我们证明了我们的算法,我们称之为杂散算法的广泛适用性和实用性。我们还演示了如何使用特征工程,该算法可以帮助检测其他数据结构中存在的异常。我们展示了杂散算法在精度和计算时间方面优于HDoutliers算法的情况。这个框架是在开源的R包stray中实现的。

英文摘要:

The HDoutliers algorithm is a powerful unsupervised algorithm for detecting anomalies in high-dimensional data, with a strong theoretical foundation. However, it suffers from some limitations that significantly hinder its performance level, under certain circumstances. In this article, we propose an algorithm that addresses these limitations. We define an anomaly as an observation that deviates markedly from the majority with a large distance gap. An approach based on extreme value theory is used for the anomalous threshold calculation. Using various synthetic and real datasets, we demonstrate the wide applicability and usefulness of our algorithm, which we call the stray algorithm. We also demonstrate how this algorithm can assist in detecting anomalies present in other data structures using feature engineering. We show the situations where the stray algorithm outperforms the HDoutliers algorithm both in accuracy and computational time. This framework is implemented in the open source R package stray.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑