arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1202.3173cs.DScs.CCcs.DCcs.NAmath.COmath.NA
收藏

Strassen矩阵乘法的通信最优并行算法

Communication-Optimal Parallel Algorithm for Strassen's Matrix Multiplication

Grey Ballard, James Demmel, Olga Holtz, Benjamin Lipshitz, Oded Schwartz

摘要:

并行矩阵乘法是分布式高性能计算中研究最多的基本问题之一。我们得到了一个新的并行算法,该算法是基于Strassen的快速矩阵乘法和最小化通信。该算法优于所有已知的并行矩阵乘法算法,经典的和基于Strassen的,无论是渐进的,并在实践中。并行化Strassen算法的一个关键瓶颈是处理器之间的通信。Ballard,Demmel,Holtz和Schwartz(SPAA'11)使用底层计算图的扩展特性证明了这些通信成本的下限。我们的算法匹配这些下限,因此是通信最优的。它在最大可能范围内表现出完美的强缩放性。基准测试我们的实现在Cray XT4上,我们获得了超过经典和基于Strassen的算法的加速比,范围从24%到184%,对于固定的矩阵维数n=94080,其中节点的数量范围从49到7203。我们的并行化方法推广到其他快速矩阵乘法算法。

英文摘要:

Parallel matrix multiplication is one of the most studied fundamental problems in distributed and high performance computing. We obtain a new parallel algorithm that is based on Strassen's fast matrix multiplication and minimizes communication. The algorithm outperforms all known parallel matrix multiplication algorithms, classical and Strassen-based, both asymptotically and in practice. A critical bottleneck in parallelizing Strassen's algorithm is the communication between the processors. Ballard, Demmel, Holtz, and Schwartz (SPAA'11) prove lower bounds on these communication costs, using expansion properties of the underlying computation graph. Our algorithm matches these lower bounds, and so is communication-optimal. It exhibits perfect strong scaling within the maximum possible range. Benchmarking our implementation on a Cray XT4, we obtain speedups over classical and Strassen-based algorithms ranging from 24% to 184% for a fixed matrix dimension n=94080, where the number of nodes ranges from 49 to 7203. Our parallelization approach generalizes to other fast matrix multiplication algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑