Robust Contextual Outlier Detection:上下文满足稀疏性的地方
Robust Contextual Outlier Detection: Where Context Meets Sparsity
摘要:
离群值检测是一项基本的数据科学任务,其应用范围从数据清理到网络安全。鉴于这项任务的基本性质,这一直是许多研究的主题。最近,出现了一类新的离群值检测算法,称为{ \it上下文离群值检测},并且在研究特定上下文中的异常行为时显示出改进的性能。然而,正如我们在本文中指出的那样,这种方法在上下文稀疏(即缺乏合适的参考框架)的情况下具有有限的适用性。此外,迄今为止开发的方法不能扩展到大型数据集。为了解决这些问题,在这里,我们提出了一种新的和强大的方法替代国家的最先进的称为鲁棒上下文离群值检测(ROCOD)。我们利用基于相关上下文的本地和全局行为模型,然后以自然和健壮的方式进行集成。我们还提出了几个优化,以提高该方法的可扩展性。我们在合成和真实世界的数据集上运行ROCOD,并证明它在功效和效率方面优于其他竞争基线(与现代上下文离群值检测方法相比,加速40倍)。我们还向下钻取并执行细粒度的分析,以阐明ROCOD的性能增益的基本原理,并揭示其在处理具有稀疏上下文的对象时的有效性。
英文摘要:
Outlier detection is a fundamental data science task with applications ranging from data cleaning to network security. Given the fundamental nature of the task, this has been the subject of much research. Recently, a new class of outlier detection algorithms has emerged, called {\it contextual outlier detection}, and has shown improved performance when studying anomalous behavior in a specific context. However, as we point out in this article, such approaches have limited applicability in situations where the context is sparse (i.e. lacking a suitable frame of reference). Moreover, approaches developed to date do not scale to large datasets. To address these problems, here we propose a novel and robust approach alternative to the state-of-the-art called RObust Contextual Outlier Detection (ROCOD). We utilize a local and global behavioral model based on the relevant contexts, which is then integrated in a natural and robust fashion. We also present several optimizations to improve the scalability of the approach. We run ROCOD on both synthetic and real-world datasets and demonstrate that it outperforms other competitive baselines on the axes of efficacy and efficiency (40X speedup compared to modern contextual outlier detection methods). We also drill down and perform a fine-grained analysis to shed light on the rationale for the performance gains of ROCOD and reveal its effectiveness when handling objects with sparse contexts.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。