arXiv 1708.074812017-08-21cs.MScs.DCcs.DSstat.COstat.ML

随机块划分流图挑战的预条件谱聚类

Preconditioned Spectral Clustering for Stochastic Block Partition Streaming Graph Challenge

David Zhuzhunashvili, Andrew Knyazev

备注：: 6 pages. To appear in Proceedings of the 2017 IEEE High Performance Extreme Computing Conference. Student Innovation Award Streaming Graph Challenge: Stochastic Block Partition, see http://graphchallenge.mit.edu/champions
发表信息：: 2017 IEEE High Performance Extreme Computing Conference (HPEC), Waltham, MA, USA, 2017, pp. 1-6
DOI：: 10.1109/HPEC.2017.8091045

摘要：

局部最优块预条件共轭梯度（LOBPCG）被证明可以有效地解决谱聚类中出现的图拉普拉斯算子的特征值问题。对于静态图分区，在没有预处理的情况下，LOBPCG的10-20次迭代导致约10倍的错误减少，足以实现具有已知真值分区的所有挑战数据集的100%正确性，例如，对于具有5 K/.1M（50 K/1 M）顶点/边的图形，只需2（7）秒，而基线Python代码需要超过5，000（30，000）秒。我们的Python代码使用10 GB（50 GB）内存在270（1，700）秒内从具有0.5M（2 M）顶点的Challenge静态图中100%正确地确定了98（160）个聚类。我们的单精度MATLAB代码在一半的时间和内存中计算相同的聚类。对于流图分区，LOBPCG是用已经为先前图计算的图拉普拉斯算子的近似特征向量来启动的，与静态情况相比，在许多情况下减少了所需LOBPCG迭代次数的2-3倍。我们的谱聚类是通用的，即假设没有特定的块模型或流，用于生成挑战的图形，与基本代码相反。尽管如此，在与5 K图的基本代码的10阶段流比较中，我们的集群的质量从阶段4（7）开始类似或更好，用于新兴边缘（滚雪球）流，而计算速度快了100-1000。

英文摘要：

Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) is demonstrated to efficiently solve eigenvalue problems for graph Laplacians that appear in spectral clustering. For static graph partitioning, 10-20 iterations of LOBPCG without preconditioning result in ~10x error reduction, enough to achieve 100% correctness for all Challenge datasets with known truth partitions, e.g., for graphs with 5K/.1M (50K/1M) Vertices/Edges in 2 (7) seconds, compared to over 5,000 (30,000) seconds needed by the baseline Python code. Our Python code 100% correctly determines 98 (160) clusters from the Challenge static graphs with 0.5M (2M) vertices in 270 (1,700) seconds using 10GB (50GB) of memory. Our single-precision MATLAB code calculates the same clusters at half time and memory. For streaming graph partitioning, LOBPCG is initiated with approximate eigenvectors of the graph Laplacian already computed for the previous graph, in many cases reducing 2-3 times the number of required LOBPCG iterations, compared to the static case. Our spectral clustering is generic, i.e. assuming nothing specific of the block model or streaming, used to generate the graphs for the Challenge, in contrast to the base code. Nevertheless, in 10-stage streaming comparison with the base code for the 5K graph, the quality of our clusters is similar or better starting at stage 4 (7) for emerging edging (snowballing) streaming, while the computations are over 100-1000 faster.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。