arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1708.08813cs.LG
收藏

异常检测:回顾和初步熵方法测试

Anomaly Detection: Review and preliminary Entropy method tests

Pelumi Oluwasanya

摘要:

异常是奇怪的数据点;它们通常代表不寻常的事件。从无线传感器网络的角度研究异常检测问题。正如我们将看到的,过去已经采取了不同的方法,不仅用于识别异常值,而且用于建立不同方法的统计特性。通常的目标是证明该方法是渐近有效的,并且所使用的度量是无偏的或可能有偏的。这个项目是基于[1]所做的工作。该方法基于当测量到异常数据点时数据的熵增加的原理。因此,要估计数据集的熵。然而,在本报告中,提出了确认[1]结果的初步努力。为了估计数据集的熵,由于没有假设参数形式,首先使用数据分裂方法估计数据集的概率密度函数。然后将该估计的pdf值插入熵估计公式以估计数据集的熵。本报告中使用的数据(测试信号)为高斯分布,均值为零,方差为4。PDF估计的结果,使用k-最近邻方法,使用整个数据集,和数据分裂的方法,并根据他们如何近似高斯的概率密度函数与类似的均值和方差进行比较。为本报告的目的选择的最近邻的数目是8。这是任意的,但是是合理的,因为在数据分割时,引入的异常数量预计将小于此。数据分割方法是首选的,也是正确的。

英文摘要:

Anomalies are strange data points; they usually represent an unusual occurrence. Anomaly detection is presented from the perspective of Wireless sensor networks. Different approaches have been taken in the past, as we will see, not only to identify outliers, but also to establish the statistical properties of the different methods. The usual goal is to show that the approach is asymptotically efficient and that the metric used is unbiased or maybe biased. This project is based on a work done by [1]. The approach is based on the principle that the entropy of the data is increased when an anomalous data point is measured. The entropy of the data set is thus to be estimated. In this report however, preliminary efforts at confirming the results of [1] is presented. To estimate the entropy of the dataset, since no parametric form is assumed, the probability density function of the data set is first estimated using data split method. This estimated pdf value is then plugged-in to the entropy estimation formula to estimate the entropy of the dataset. The data (test signal) used in this report is Gaussian distributed with zero mean and variance 4. Results of pdf estimation using the k-nearest neighbour method using the entire dataset, and a data-split method are presented and compared based on how well they approximate the probability density function of a Gaussian with similar mean and variance. The number of nearest neighbours chosen for the purpose of this report is 8. This is arbitrary, but is reasonable since the number of anomalies introduced is expected to be less than this upon data-split. The data-split method is preferred and rightly so.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑