基于Cluster Catch Digraphs(聚类捕获有向图)的异常值检测
Outlier Detection with Cluster Catch Digraphs
AI总结:
针对高维、聚类形状多变导致传统异常值检测性能下降的问题,提出四类基于聚类捕获有向图的新型算法,经模拟与真实数据集验证,在准确性、适应性上有显著提升。
摘要:
本文介绍了一类基于Cluster Catch Digraphs(CCDs,聚类捕获有向图)的新型异常值检测算法,专门用于应对高维度和聚类形状多变带来的挑战——这两类问题会导致大多数传统异常值检测方法的性能下降。我们提出了基于均匀性的互捕获图CCD(U-MCCD)、基于均匀性与邻域的互捕获图CCD(UN-MCCD),以及它们的形状自适应变体(SU-MCCD和SUN-MCCD),这些算法旨在检测具有任意聚类形状的高维数据集中的异常值。我们阐述了这些算法的优缺点,并说明了定义每一种特定算法的动因与必要性。通过全面的蒙特卡洛模拟,我们评估了它们的性能,证明了我们的算法在不同设置和污染水平下均具备鲁棒性与有效性。我们还在多个真实数据集上演示了这些算法的应用。U-MCCD算法能高效识别异常值,同时保持较高的真负率;SU-MCCD算法在处理非均匀聚类方面表现出显著提升。此外,UN-MCCD和SUN-MCCD算法通过将最近邻距离(NND)用于聚类与异常值检测,解决了现有方法在高维空间中的局限性。我们的结果表明,这些新型算法在异常值检测的准确性与适应性方面实现了重大突破,可为各类现实应用提供有价值的工具。
英文摘要:
This paper introduces a novel family of outlier detection algorithms based on Cluster Catch Digraphs (CCDs), specifically tailored to address the challenges of high dimensionality and varying cluster shapes, which deteriorate the performance of most traditional outlier detection methods. We propose the Uniformity-Based CCD with Mutual Catch Graph (U-MCCD), the Uniformity- and Neighbor-Based CCD with Mutual Catch Graph (UN-MCCD), and their shape-adaptive variants (SU-MCCD and SUN-MCCD), which are designed to detect outliers in data sets with arbitrary cluster shapes and high dimensions. We present the advantages and shortcomings of these algorithms and provide the motivation or need to define each particular algorithm. Through comprehensive Monte Carlo simulations, we assess their performance and demonstrate the robustness and effectiveness of our algorithms across various settings and contamination levels. We also illustrate the use of our algorithms on various real-life data sets. The U-MCCD algorithm efficiently identifies outliers while maintaining high true negative rates, and the SU-MCCD algorithm shows substantial improvement in handling non-uniform clusters. Additionally, the UN-MCCD and SUN-MCCD algorithms address the limitations of existing methods in high-dimensional spaces by utilizing Nearest Neighbor Distances (NND) for clustering and outlier detection. Our results indicate that these novel algorithms offer substantial advancements in the accuracy and adaptability of outlier detection, providing a valuable tool for various real-world applications. Keyword: Outlier detection, Graph-based clustering, Cluster catch digraphs, $k$-nearest-neighborhood, Mutual catch graphs, Nearest neighbor distance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。