arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.15874cs.LGcs.AI
收藏

异常值分数的鲁棒统计缩放:提升异常值的异常概率质量(扩展版)

Robust Statistical Scaling of Outlier Scores: Improving the Quality of Outlier Probabilities for Outliers (Extended Version)

Johannes Gutenberg University Mainz(美因茨约翰内斯古滕贝格大学);University of Southern Denmark(南丹麦大学)

Philipp Röchner, Henrique O. Marques, Ricardo J. G. B. Campello, Arthur Zimek, Franz Rothlauf

AI总结:

针对现有统计缩放方法对异常值概率估计质量不佳的问题,提出采用鲁棒估计器的鲁棒统计缩放方法,经真实数据集与多种算法验证,可有效提升异常值的概率估计质量。

摘要:

异常检测算法通常会为数据集中的每个观测值分配一个异常值分数,用于表示该观测值属于异常值的程度。然而,这些分数往往无法在不同算法间进行比较,且人类难以解读。统计缩放方法通过无需真实标签即可将异常值分数转换为异常概率的方式解决了这一问题,从而提升了可解释性与跨算法的可比性。不过,这种转换对异常值和正常样本的质量可能存在差异。在医疗、金融或工程等对异常值关注度极高的场景中,漏检异常值可能会造成高昂代价或引发危险。因此,确保异常值的概率估计质量至关重要。本文指出,文献中常用的统计缩放方法对异常值生成的概率质量不如对正常样本的。为此,我们提出鲁棒统计缩放方法,该方法采用鲁棒估计器来提升异常值的概率质量。我们在真实数据集和多种异常检测算法上,将该方法的多个变体与其他异常值分数转换方法进行了对比评估,结果表明该方法能够提升异常值的概率估计质量。

英文摘要:

Outlier detection algorithms typically assign an outlier score to each observation in a dataset, indicating the degree to which an observation is an outlier. However, these scores are often not comparable across algorithms and can be difficult for humans to interpret. Statistical scaling addresses this problem by transforming outlier scores into outlier probabilities without using ground-truth labels, thereby improving interpretability and comparability across algorithms. However, the quality of this transformation can be different for outliers and inliers. Missing outliers in scenarios where they are of particular interest - such as healthcare, finance, or engineering - can be costly or dangerous. Thus, ensuring good probabilities for outliers is essential. This paper argues that statistical scaling, as commonly used in the literature, does not produce equally good probabilities for outliers as for inliers. Therefore, we propose robust statistical scaling, which uses robust estimators to improve the probabilities for outliers. We evaluate several variants of our method against other outlier score transformations for real-world datasets and outlier detection algorithms, where it can improve the probabilities for outliers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑