线性代数中的最小化通信
Minimizing Communication in Linear Algebra
摘要：
1981年，洪和孔证明了使用传统的$O(n^3)$算法执行密集矩阵乘法所需的通信量的下限，其中输入矩阵太大而无法容纳在小而快的存储器中。在2004年Irony，Toledo和Tiskin给出了这个结果的一个新的证明，并将其推广到平行情形。在这两种情况下，下限可以表示为$Ω$（算术运算数/ $\sqrt{M}$），其中M是快速存储器（或并行情况下的本地存储器）的大小。在这里，我们将这些结果推广到更广泛的算法，包括LU分解，Cholesky分解，$LDL^T$分解，QR分解，特征值和奇异值的算法，即，线性代数的所有直接方法。该证明适用于稠密或稀疏矩阵，以及顺序或并行算法。除了移动的数据量（带宽）的下限之外，我们还得到了移动数据所需的消息数量（延迟）的下限。我们说明了如何将我们的下界技术扩展到线性代数运算的组合（如矩阵的计算能力），以决定是否足以调用一系列更简单的最佳算法（如矩阵乘法）来最大限度地减少通信，或者我们是否可以做得更好。我们举两个例子。我们还展示了如何将我们的下界扩展到某些图论问题。我们指出最近设计的算法密集LU，Cholesky，QR，特征值和SVD问题，达到这些下限; LU和QR的实现显示出大的加速比传统的线性代数算法在标准库，如LAPACK和ScaLAPACK。许多悬而未决的问题仍然存在。
英文摘要：
In 1981 Hong and Kung proved a lower bound on the amount of communication needed to perform dense, matrix-multiplication using the conventional $O(n^3)$ algorithm, where the input matrices were too large to fit in the small, fast memory. In 2004 Irony, Toledo and Tiskin gave a new proof of this result and extended it to the parallel case. In both cases the lower bound may be expressed as $Ω$(#arithmetic operations / $\sqrt{M}$), where M is the size of the fast memory (or local memory in the parallel case). Here we generalize these results to a much wider variety of algorithms, including LU factorization, Cholesky factorization, $LDL^T$ factorization, QR factorization, algorithms for eigenvalues and singular values, i.e., essentially all direct methods of linear algebra. The proof works for dense or sparse matrices, and for sequential or parallel algorithms. In addition to lower bounds on the amount of data moved (bandwidth) we get lower bounds on the number of messages required to move it (latency). We illustrate how to extend our lower bound technique to compositions of linear algebra operations (like computing powers of a matrix), to decide whether it is enough to call a sequence of simpler optimal algorithms (like matrix multiplication) to minimize communication, or if we can do better. We give examples of both. We also show how to extend our lower bounds to certain graph theoretic problems. We point out recently designed algorithms for dense LU, Cholesky, QR, eigenvalue and the SVD problems that attain these lower bounds; implementations of LU and QR show large speedups over conventional linear algebra algorithms in standard libraries like LAPACK and ScaLAPACK. Many open problems remain.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
