Learning the community structure of a large-scale graph is a fundamental problem in machine learning, computer science and statistics. We study the problem of exactly recovering the communities in a graph generated from the Stochastic Block Model (SBM) in the Massively Parallel Computation (MPC) model. Specifically, given $kn$ vertices that are partitioned into $k$ equal-sized clusters (i.e., each has size $n$), a graph on these $kn$ vertices is randomly generated such that each pair of vertices is connected with probability~$p$ if they are in the same cluster and with probability $q$ if not, where $p > q > 0$. We give MPC algorithms for the SBM in the (very general) \emph{$s$-space MPC model}, where each machine has memory $s=Ω(\log n)$. Under the condition that $\frac{p-q}{\sqrt{p}}\geq \tildeΩ(k^{\frac12}n^{-\frac12+\frac{1}{2(r-1)}})$ for any integer $r\in [3,O(\log n)]$, our first algorithm exactly recovers all the $k$ clusters in $O(kr\log_s n)$ rounds using $\tilde{O}(m)$ total space, or in $O(r\log_s n)$ rounds using $\tilde{O}(km)$ total space. If $\frac{p-q}{\sqrt{p}}\geq \tildeΩ(k^{\frac34}n^{-\frac14})$, our second algorithm achieves $O(\log_s n)$ rounds and $\tilde{O}(m)$ total space complexity. Both algorithms significantly improve upon a recent result of Cohen-Addad et al. [PODC'22], who gave algorithms that only work in the \emph{sublinear space MPC model}, where each machine has local memory~$s=O(n^δ)$ for some constant $δ>0$, with a much stronger condition on $p,q,k$.
Our algorithms are based on collecting the $r$-step neighborhood of each vertex and comparing the difference of some statistical information generated from the local neighborhoods for each pair of vertices. To implement the clustering algorithms in parallel, we present efficient approaches for implementing some basic graph operations in the $s$-space MPC model.
Improved Deterministic Connectivity in Massively Parallel Computation
关于大规模并行计算模型中的连接性的一系列研究在Andoni等人的开创性工作中达到了高潮。[FOCS'18]和Behnezhad等人。[FOCS'19]。他们提供了一个低空间MPC的随机算法，并证明是最佳的轮复杂度$O(\log D + \log \log_{\frac m n} n)$和$O(m)$空间，对于图的$n$顶点与$m$边和直径$D$。令人惊讶的是，Coy和Czumaj [STOC'22]最近的结果显示了如何确定性地实现相同的结果。然而，不幸的是，他们的算法遭受大的本地计算时间。我们提出了一个确定性的连接算法，匹配的所有参数的随机算法，此外，显着减少了本地计算时间接近线性。我们的去随机化方法是基于减少所需的随机性，以允许一个更简单的高效搜索。虽然类似的随机性减少方法已经使用过，我们的结果不仅是惊人的简单，但它是第一个有效率的本地计算。这就是为什么我们认为它可以作为系统开发低内存MPC中计算高效的去随机化方法的起点。
