arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.14730cs.DScs.CCcs.DC
收藏

平均连接法难以实现层次凝聚聚类!

It's Hard to HAC with Average Linkage!

MohammadHossein Bateni, Laxman Dhulipala, Kishen N Gowda, D Ellis Hershkowitz, Rajesh Jayaram, Jakub Łącki

AI总结:

本文证明平均连接层次凝聚聚类在串行和并行计算上均存在固有困难,并指出其在路径图及低输出层次高度情形下仍可高效求解。

摘要:

平均连接层次凝聚聚类(HAC)是一种被广泛研究和应用的层次聚类方法。近年来,针对大规模数据集的应用推动了对平均连接HAC的近线性时间算法和高效并行算法的极大关注。我们提供了否定性的困难度结果,排除了这类算法的存在。在串行方面,我们在标准细粒度复杂性假设下,针对n个节点的图建立了串行组合算法的运行时间下界n^{3/2-ε}。这基本上与平均连接HAC目前已知的最佳运行时间相匹配。在并行方面,我们通过证明平均连接HAC在直径为4的树上具有CC困难性,表明即使在简单图上,平均连接HAC也可能无法并行化。在可行性方面,我们证明了平均连接HAC在路径上可以被高效并行化(即属于NC类),并且当输出聚类层次结构的高度较小时,可以在近线性时间内求解。

英文摘要:

Average linkage Hierarchical Agglomerative Clustering (HAC) is an extensively studied and applied method for hierarchical clustering. Recent applications to massive datasets have driven significant interest in near-linear-time and efficient parallel algorithms for average linkage HAC. We provide hardness results that rule out such algorithms. On the sequential side, we establish a runtime lower bound of $n^{3/2-ε}$ on $n$ node graphs for sequential combinatorial algorithms under standard fine-grained complexity assumptions. This essentially matches the best-known running time for average linkage HAC. On the parallel side, we prove that average linkage HAC likely cannot be parallelized even on simple graphs by showing that it is CC-hard on trees of diameter $4$. On the possibility side, we demonstrate that average linkage HAC can be efficiently parallelized (i.e., it is in NC) on paths and can be solved in near-linear time when the height of the output cluster hierarchy is small.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑