We study the known techniques for designing Matrix Multiplication algorithms. The two main approaches are the Laser method of Strassen, and the Group theoretic approach of Cohn and Umans. We define a generalization based on zeroing outs which subsumes these two approaches, which we call the Solar method, and an even more general method based on monomial degenerations, which we call the Galactic method.
We then design a suite of techniques for proving lower bounds on the value of $ω$, the exponent of matrix multiplication, which can be achieved by algorithms using many tensors $T$ and the Galactic method. Some of our techniques exploit `local' properties of $T$, like finding a sub-tensor of $T$ which is so `weak' that $T$ itself couldn't be used to achieve a good bound on $ω$, while others exploit `global' properties, like $T$ being a monomial degeneration of the structural tensor of a group algebra.
Our main result is that there is a universal constant $\ell>2$ such that a large class of tensors generalizing the Coppersmith-Winograd tensor $CW_q$ cannot be used within the Galactic method to show a bound on $ω$ better than $\ell$, for any $q$. We give evidence that previous lower-bounding techniques were not strong enough to show this. We also prove a number of complementary results along the way, including that for any group $G$, the structural tensor of $\mathbb{C}[G]$ can be used to recover the best bound on $ω$ which the Coppersmith-Winograd approach gets using $CW_{|G|-2}$ as long as the asymptotic rank of the structural tensor is not too large.
Faster Algorithms for Rectangular Matrix Multiplication
设α是最大值,使得n × n^α矩阵与n^α × n矩阵的乘积可以用n^{2+o(1)}次算术运算来计算。本文证明了α>0.30298,改进了Coppermith(Journal of Complexity,1997)的先前记录α>0.29462。更一般地,我们构造了一个新的算法,用于将n x n^k矩阵乘以n^k x n矩阵,对于任何值k \neq 1。该算法的复杂度优于所有已知的矩形矩阵乘法算法。在方矩阵乘法的情况下(即,对于k=1),我们精确地恢复了Coppermith和Winograd(Journal of符号计算,1990)的算法的复杂性。这些新的上界可以用来改善依赖于矩形矩阵乘法的几个已知算法的时间复杂度。例如,我们直接得到了小整数权值有向图上所有对最短路径问题的O(n^{2.5302})时间算法,改进了Zwick(JACM 2002)的O(n^{2.575})时间算法,同时也改进了稀疏方阵乘法的时间复杂度.
Strong Scaling of Matrix Multiplication Algorithms and Memory-Independent Communication Lower Bounds
如果并行算法在P个处理机上的运行时间是1/P线性的,包括所有的通信开销,则该并行算法具有完美的强标度性。具有完美强标度的矩阵乘法的分布式内存并行算法最近才被发现。一种是基于经典矩阵乘法(Solomonik and Demmel,2011),另一种是基于Strassen的快速矩阵乘法(Ballard,Demmel,Holtz,Lipshitz,and Schwartz,2012)。这两种算法都可以完美地扩展,但只能扩展到处理器间通信不再扩展的处理器数量。我们得到了一个内存独立的通信成本的经典和基于Strassen的分布式存储矩阵乘法算法的下界。这些界限意味着,没有经典的或基于Strassen的并行矩阵乘法算法可以强大的规模完全超出上述两个并行算法已经达到的范围。记忆无关的边界和强缩放边界推广到其他算法。