arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2009.03117stat.MEmath.STstat.TH
收藏

针对大量数据流的异常检测:一种基于排列的更高批评方法

Anomaly Detection for a Large Number of Streams: A Permutation-Based Higher Criticism Approach

Technische Universiteit Eindhoven(埃因霍温理工大学);University of California, San Diego(加州大学圣迭戈分校)

Ivo V. Stoepker, Rui M. Castro, Ery Arias-Castro, Edwin van den Heuvel

AI总结:

本文提出了一种基于排列的更高批评统计量,用于在有限样本中实现精确的异常检测,适用于高维数据流,且在指数模型中渐近最优。

摘要:

在观察大量数据流时进行异常检测在多种应用中至关重要,从流行病学研究到复杂系统的监控。高维场景通常通过扫描统计法及相关方法来处理,需要严格的建模假设以进行正确校准。在本文中,我们采取非参数方法,提出了一种基于排列的更高批评统计量的变体,无需了解空假设分布。这导致在有限样本中具有精确检验,且在指数模型的广泛类别中渐近最优。我们展示了在有限样本中功率损失相对于理想检验是极小的。此外,由于所提出的统计量不依赖渐近近似,因此通常比依赖此类近似的更高批评变体表现更好。我们包括了推荐,使该检验可以方便地应用于实际,并展示了其在监控批量生产药物产品活性成分内容均匀性方面的适用性。

英文摘要:

Anomaly detection when observing a large number of data streams is essential in a variety of applications, ranging from epidemiological studies to monitoring of complex systems. High-dimensional scenarios are usually tackled with scan-statistics and related methods, requiring stringent modeling assumptions for proper calibration. In this work we take a non-parametric stance, and propose a permutation-based variant of the higher criticism statistic not requiring knowledge of the null distribution. This results in an exact test in finite samples which is asymptotically optimal in the wide class of exponential models. We demonstrate the power loss in finite samples is minimal with respect to the oracle test. Furthermore, since the proposed statistic does not rely on asymptotic approximations it typically performs better than popular variants of higher criticism that rely on such approximations. We include recommendations such that the test can be readily applied in practice, and demonstrate its applicability in monitoring the content uniformity of an active ingredient for a batch-produced drug product.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑