红蓝pebbling再访：近最优并行矩阵-矩阵乘法
Red-blue pebbling revisited: near optimal parallel matrix-matrix multiplication
摘要：
我们建议宇宙空间协会：一种并行矩阵-矩阵乘法算法，对于矩阵维度、处理器计数和存储器大小的所有组合而言，该算法接近通信最优。COSMA背后的关键思想是导出一个最佳的（对于10 MB的快速内存，最高可达0.03\%）顺序调度，然后将其并行化，保持I/O最优性。为了实现这一点，我们使用红蓝卵石游戏精确建模MMM依赖，并推导出一个建设性的和紧密的顺序和并行I/O下界证明。与2D或3D算法（预先修复处理器分解，然后将其映射到矩阵维度）相比，它将通信量减少了高达$\sqrt{3}$倍。COSMA在所有场景中的性能都优于ScaLAPACK、CARMA和CTF算法，最高可达12.8倍（平均2.2倍），达到Piz Daint峰值性能的88%。我们的工作不需要任何手动调优，并且作为开源实现进行维护。
英文摘要：
We propose COSMA: a parallel matrix-matrix multiplication algorithm that is near communication-optimal for all combinations of matrix dimensions, processor counts, and memory sizes. The key idea behind COSMA is to derive an optimal (up to a factor of 0.03\% for 10MB of fast memory) sequential schedule and then parallelize it, preserving I/O optimality. To achieve this, we use the red-blue pebble game to precisely model MMM dependencies and derive a constructive and tight sequential and parallel I/O lower bound proofs. Compared to 2D or 3D algorithms, which fix processor decomposition upfront and then map it to the matrix dimensions, it reduces communication volume by up to $\sqrt{3}$ times. COSMA outperforms the established ScaLAPACK, CARMA, and CTF algorithms in all scenarios up to 12.8x (2.2x on average), achieving up to 88\% of Piz Daint's peak performance. Our work does not require any hand tuning and is maintained as an open source implementation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
