McCatch：维度和非维度数据集中的可扩展微簇检测
McCatch: Scalable Microcluster Detection in Dimensional and Nondimensional Datasets
AI总结：
本文提出McCatch算法，利用Oracle图（1NN距离与组1NN距离）在维度和非维度数据中检测微簇，能同时排序单例和非单例异常，优于11种方法，并在多种数据上验证有效性。
摘要：
我们如何能有一个异常检测器，即使面对非维度数据也能工作，并同时将单例微簇（“一次性”异常）和非单例微簇按其异常分数进行排序？如何以一种可扩展且“无需人工干预”的方式获得有原则的分数？异常微簇表明欺诈活动中的联合或重复等；因此，识别它们是非常可取的。本文提出了McCatch：一种新算法，通过利用我们提出的“Oracle”图（1NN距离与组1NN距离）来检测微簇。我们研究了31个真实和合成数据集，包含多达100万个数据元素，以表明McCatch是唯一能回答上述两个问题的方法；并且，它优于其他11种方法，尤其是在数据具有非单例微簇或非维度的情况下。我们还展示了McCatch在检测图、指纹、网络连接日志、文本数据和卫星图像中有意义的微簇的能力。例如，它在网络日志中发现了包含30个元素的已确认“拒绝服务”攻击的微簇，在一台普通台式机上处理222K个数据元素仅需约3分钟。
英文摘要：
How could we have an outlier detector that works even with nondimensional data, and ranks together both singleton microclusters ('one-off' outliers) and nonsingleton microclusters by their anomaly scores? How to obtain scores that are principled in one scalable and 'hands-off' manner? Microclusters of outliers indicate coalition or repetition in fraud activities, etc.; their identification is thus highly desirable. This paper presents McCatch: a new algorithm that detects microclusters by leveraging our proposed 'Oracle' plot (1NN Distance versus Group 1NN Distance). We study 31 real and synthetic datasets with up to 1M data elements to show that McCatch is the only method that answers both of the questions above; and, it outperforms 11 other methods, especially when the data has nonsingleton microclusters or is nondimensional. We also showcase McCatch's ability to detect meaningful microclusters in graphs, fingerprints, logs of network connections, text data, and satellite imagery. For example, it found a 30-elements microcluster of confirmed 'Denial of Service' attacks in the network logs, taking only ~3 minutes for 222K data elements on a stock desktop.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
