arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1708.09441cs.LGcs.AIstat.ML
收藏

将反馈引入基于树的异常检测

Incorporating Feedback into Tree-based Anomaly Detection

Oregon State University(俄勒冈州立大学)

Shubhomoy Das, Weng-Keen Wong, Alan Fern, Thomas G. Dietterich, Md Amran Siddiqui

摘要:

异常检测器通常用于生成统计异常的排名列表,由人类分析师检查,以提取感兴趣的实际异常。不幸的是,在现实世界的应用程序中,这个过程对分析师来说可能非常困难,因为大部分高级别的异常都是误报,从应用程序的角度来看并不有趣。在本文中,我们的目标是使分析师的工作更容易,允许在调查过程中的分析师反馈。理想地,反馈以减少在发现感兴趣的异常之前必须检查的误报的数量的方式影响异常检测器的排名。特别是,我们介绍了一种新的技术,将简单的二进制反馈到基于树的异常检测器。我们专注于隔离森林算法作为一个代表性的基于树的异常检测器,并表明,我们可以显着提高其性能,通过将反馈,与基线算法相比,不纳入反馈。我们的技术是简单的,规模以及数据的大小增加,这使得它适合在大型数据集的异常交互式发现。

英文摘要:

Anomaly detectors are often used to produce a ranked list of statistical anomalies, which are examined by human analysts in order to extract the actual anomalies of interest. Unfortunately, in realworld applications, this process can be exceedingly difficult for the analyst since a large fraction of high-ranking anomalies are false positives and not interesting from the application perspective. In this paper, we aim to make the analyst's job easier by allowing for analyst feedback during the investigation process. Ideally, the feedback influences the ranking of the anomaly detector in a way that reduces the number of false positives that must be examined before discovering the anomalies of interest. In particular, we introduce a novel technique for incorporating simple binary feedback into tree-based anomaly detectors. We focus on the Isolation Forest algorithm as a representative tree-based anomaly detector, and show that we can significantly improve its performance by incorporating feedback, when compared with the baseline algorithm that does not incorporate feedback. Our technique is simple and scales well as the size of the data increases, which makes it suitable for interactive discovery of anomalies in large datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑