arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.08212cs.LGcs.AI
收藏

D.MCA:基于显式微簇聚类的离群点检测

D.MCA: Outlier Detection with Explicit Micro-Cluster Assignments

Carnegie Mellon University(卡内基梅隆大学);University of São Paulo(圣保罗大学)

Shuli Jiang, Robson Leonardo Ferreira Cordeiro, Leman Akoglu

摘要:

我们如何检测离群值,无论是分散的还是聚集的,并且还明确地将它们分配到相应的微集群,而不知道先验有多少微集群存在?我们如何在内部执行这两项任务,即,没有任何事后处理,使检测和分配可以同时受益于彼此?在许多实际应用中,在单独的微集群中呈现离群值对分析师来说是信息丰富的。然而,基于由任何现有方法检测到的离群值的事后聚类的天真解决方案具有两个主要缺点:(a)对于聚类来说,适当的超参数值通常是未知的,并且大多数算法都难以处理不同形状和密度的聚类;(b)检测和分配不能相互受益。在本文中,我们提出了D.MCA到$\underline{D}$检测离群值与明确的$\underline{M}$ icro- $\underline{C}$光泽$\underline{A}$ signment。我们的方法通过使用一种新的策略来迭代地执行检测和分配,该策略从训练集中修剪整个微簇以提高检测的性能。它还受益于一种新的策略,该策略避免了聚集的离群值相互掩盖,这是文献中众所周知的问题。此外,D.MCA被设计为通过采用超强度“预热”阶段对临界超参数具有鲁棒性。在16个真实世界和合成数据集上进行的实验表明,D.MCA优于8个最先进的竞争对手,特别是在显式离群微聚类分配任务上。

英文摘要:

How can we detect outliers, both scattered and clustered, and also explicitly assign them to respective micro-clusters, without knowing apriori how many micro-clusters exist? How can we perform both tasks in-house, i.e., without any post-hoc processing, so that both detection and assignment can benefit simultaneously from each other? Presenting outliers in separate micro-clusters is informative to analysts in many real-world applications. However, a naïve solution based on post-hoc clustering of the outliers detected by any existing method suffers from two main drawbacks: (a) appropriate hyperparameter values are commonly unknown for clustering, and most algorithms struggle with clusters of varying shapes and densities; (b) detection and assignment cannot benefit from one another. In this paper, we propose D.MCA to $\underline{D}$etect outliers with explicit $\underline{M}$icro-$\underline{C}$luster $\underline{A}$ssignment. Our method performs both detection and assignment iteratively, and in-house, by using a novel strategy that prunes entire micro-clusters out of the training set to improve the performance of the detection. It also benefits from a novel strategy that avoids clustered outliers to mask each other, which is a well-known problem in the literature. Also, D.MCA is designed to be robust to a critical hyperparameter by employing a hyperensemble "warm up" phase. Experiments performed on 16 real-world and synthetic datasets demonstrate that D.MCA outperforms 8 state-of-the-art competitors, especially on the explicit outlier micro-cluster assignment task.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑