我们按照 [D. Bini 和 G. Lotti, 快速矩阵乘法算法的稳定性, Numer. Math. 36 (1980), 63--72] 的精神，对一大类递归矩阵乘法算法进行了前向误差分析。作为分析的结果，我们证明了矩阵乘法的指数（最优运行时间）可以通过数值稳定的算法实现。我们还证明了 [H. Cohn 和 C. Umans, 一种群论方法实现快速矩阵乘法, FOCS 2003, 438--449] 和 [H. Cohn, R. Kleinberg, B. Szegedy 和 C. Umans, 群论算法用于矩阵乘法, FOCS 2005, 379--388] 中提出的新群论算法都包含在我们的分析适用的算法类别中，因此是数值稳定的。我们对三种特定的快速群论算法进行了详细的误差分析。
英文摘要：
We perform forward error analysis for a large class of recursive matrix multiplication algorithms in the spirit of [D. Bini and G. Lotti, Stability of fast algorithms for matrix multiplication, Numer. Math. 36 (1980), 63--72]. As a consequence of our analysis, we show that the exponent of matrix multiplication (the optimal running time) can be achieved by numerically stable algorithms. We also show that new group-theoretic algorithms proposed in [H. Cohn, and C. Umans, A group-theoretic approach to fast matrix multiplication, FOCS 2003, 438--449] and [H. Cohn, R. Kleinberg, B. Szegedy and C. Umans, Group-theoretic algorithms for matrix multiplication, FOCS 2005, 379--388] are all included in the class of algorithms to which our analysis applies, and are therefore numerically stable. We perform detailed error analysis for three specific fast group-theoretic algorithms.
We study the properties of ultrametric matrices aiming to design methods for fast ultrametric matrix-vector multiplication. We show how to encode such a matrix as a tree structure in quadratic time and demonstrate how to use the resulting representation to perform matrix-vector multiplications in linear time. Accompanying this article, we provide an implementation of the proposed algorithms and present empirical results on their practical performance.
We present two fast algorithms for matrix-vector multiplication $y=Ax$, where $A$ is a Hankel matrix. The current asymptotically fastest method is based on the Fast Fourier Transform (FFT), however in multiprecision arithmetics with very high accuracy FFT method is actually slower than schoolbook multiplication for matrix sizes up to $n=8000$. One method presented is based on a decomposition of multiprecision numbers into sums, and applying standard or double precision FFT. The second method, inspired by Karatsuba multiplication, is based on recursively performing multiplications with matrices…
The communication cost of algorithms (also known as I/O-complexity) is shown to be closely related to the expansion properties of the corresponding computation graphs. We demonstrate this on Strassen's and other fast matrix multiplication algorithms, and obtain first lower bounds on their communication costs. In the sequential case, where the processor has a fast memory of size $M$, too small to store three $n$-by-$n$ matrices, the lower bound on the number of words moved between fast and slow memory is, for many of the matrix multiplication algorithms, $Ω((\frac{n}{\sqrt M})^{ω_0}\cdot M)$, w…
The article considers arrowhead and diagonal-plus-rank-one matrices in F^(nxn) where F in R,C or H. H is a non-commutative field of quaternions. We give unified formulas for fast matrix-vector multiplications, determinants, and inverses for considered matrices. The formulas are unified in the sense that the same formula holds in both, commutative and noncommutative algebras. Each formula requires O(n) arithmetic operations. Most of the formulas hold for block matrices, as well.
Invariance transformations of polyadic decompositions of matrix multiplication tensors define an equivalence relation on the set of such decompositions. In this paper, we present an algorithm to efficiently decide whether two polyadic decompositions of a given matrix multiplication tensor are equivalent. With this algorithm, we analyze the equivalence classes of decompositions of several matrix multiplication tensors. This analysis is relevant for the study of fast matrix multiplication as it relates to the question of how many essentially different fast matrix multiplication algorithms there …
Optimized multiple precision basic linear computation, especially matrix multiplication, is crucial for solving ill-conditioned problems. The recently proposed Ozaki scheme, which implements accurate matrix multiplication using existing optimized low precision matrix multiplication, is known to be useful for multiple precision as well. In this paper, we implement fixed precision multi-component-way matrix multiplication using Ozaki scheme and show that in some cases it is faster than existing optimized matrix multiplications. We also show that arbitrary precision matrix multiplication using Oz…
Can linear systems be solved faster than matrix multiplication? While there has been remarkable progress for the special cases of graph structured linear systems, in the general setting, the bit complexity of solving an $n \times n$ linear system $Ax=b$ is $\tilde{O}(n^ω)$, where $ω< 2.372864$ is the matrix multiplication exponent. Improving on this has been an open problem even for sparse linear systems with poly$(n)$ condition number. In this paper, we present an algorithm that solves linear systems in sparse matrices asymptotically faster than matrix multiplication for any $ω> 2$. This spee…
In calculating integral or discrete transforms, use has been made of fast algorithms for multiplying vectors by matrices whose elements are specified as values of special (Chebyshev, Legendre, Laguerre, etc.) functions. The currently available fast algorithms are several orders of magnitude less efficient than the fast Fourier transform. To achieve higher efficiency, a convenient general approach for calculating matrix-vector products for some class of problems is proposed. A series of fast simple-structure algorithms developed under this approach can be efficiently implemented with software b…
We consider a fast, data-sparse directional method to realize matrix-vector products related to point evaluations of the Helmholtz kernel. The method is based on a hierarchical partitioning of the point sets and the matrix. The considered directional multi-level approximation of the Helmholtz kernel can be applied even on high-frequency levels efficiently. We provide a detailed analysis of the almost linear asymptotic complexity of the presented method. Our numerical experiments are in good agreement with the provided theory.
Parallel matrix multiplication is one of the most studied fundamental problems in distributed and high performance computing. We obtain a new parallel algorithm that is based on Strassen's fast matrix multiplication and minimizes communication. The algorithm outperforms all known parallel matrix multiplication algorithms, classical and Strassen-based, both asymptotically and in practice. A critical bottleneck in parallelizing Strassen's algorithm is the communication between the processors. Ballard, Demmel, Holtz, and Schwartz (SPAA'11) prove lower bounds on these communication costs, using ex…
A theoretical contribution to the fast implementation of null linear discriminant analysis method using random matrix multiplication with scatter matrices
Jiangsu Normal University（江苏师范大学）；China University of Mining and Technology（中国矿业大学）
This paper develops fast and efficient algorithms for computing Tucker decomposition with a given multilinear rank. By combining random projection and the power scheme, we propose two efficient randomized versions for the truncated high-order singular value decomposition (T-HOSVD) and the sequentially T-HOSVD (ST-HOSVD), which are two common algorithms for approximating Tucker decomposition. To reduce the complexities of these two algorithms, fast and efficient algorithms are designed by combining two algorithms and approximate matrix multiplication. The theoretical results are also achieved b…
In this article we corrected the trilinear formula for triple disjoint matrix multiplication given in the article 'J. Laderman, V. Pan, X. H. Sha, On practical Algorithms for Accelerated Matrix Multiplication, Linear Algebra and its Applications. Vol. 162-164 (1992) pp. 557-588', which is incorrect for matrix dimensions equal to two or greater. That formula is a base of two algorithms, for disjoint and single matrix multiplication. The necessary correction made the amount of non scalar products raise, slightly increasing the algorithm time complexity. We also corrected explicit formulas, in th…
One of the most common, but at the same time expensive operations in linear algebra, is multiplying two matrices $A$ and $B$. With the rapid development of machine learning and increases in data volume, performing fast matrix intensive multiplications has become a major hurdle. Two different approaches to overcoming this issue are, 1) to approximate the product; and 2) to perform the multiplication distributively. A \textit{$CR$-multiplication} is an approximation where columns and rows of $A$ and $B$ are respectively sampled with replacement. In the distributed setting, multiple workers perfo…
