对称线性代数核的I/O优化算法
I/O-Optimal Algorithms for Symmetric Linear Algebra Kernels
摘要：
本文研究了线性代数中两个基本的对称核：Cholesky分解和对称秩$k$更新（SYRK），并给出了这两个核的经典三圈嵌套算法。此外，我们考虑一个机器模型的快速存储器的大小$S$和一个无界的慢内存。在这个模型中，所有的计算都必须在快速存储器中的操作数上执行，目标是最小化慢速和快速存储器之间的通信量。作为一组计算是固定的算法的选择，只有顺序的计算（时间表）直接影响量的communications.We证明下界的$\frac{1}{3\sqrt{2}}\frac{N^3}{\sqrt{S}}$的Cholesky分解的$N\times N$对称正定矩阵的通信量，和$\frac{1}{\sqrt{2}}\frac{N^2M}{\sqrt{S}}$的SYRK计算的$\mat{A}\cdot\transpose{\mat{A}}$，其中$\mathbf{A}$是一个$N\times M$矩阵。这两个界限提高了最知名的下限从文献中的一个因素$\sqrt{2}$。此外，我们提出了两个核心外，顺序算法与匹配的通信量：\TBS SYRK，与体积的$\frac{1}{\sqrt{2}}\frac{N^2M}{\sqrt{S}} + \bigo{NM\log N}$，和\LBC Cholesky，与体积的$\frac{1}{3\sqrt{2}}\frac{N^3}{\sqrt{S}} + \bigo{N^{5/2}}$。这两种算法都比文献中最著名的算法提高了一个因子$\sqrt{2}$，并证明了我们的下界中的领先项不能进一步改进。这项工作表明，对称内核如SYRK或Cholesky的操作强度本质上高于相应的非对称内核（GEMM和LU因式分解）（因子$\sqrt{2}$）。
英文摘要：
In this paper, we consider two fundamental symmetric kernels in linear algebra: the Cholesky factorization and the symmetric rank-$k$ update (SYRK), with the classical three nested loops algorithms for these kernels. In addition, we consider a machine model with a fast memory of size $S$ and an unbounded slow memory. In this model, all computations must be performed on operands in fast memory, and the goal is to minimize the amount of communication between slow and fast memories. As the set of computations is fixed by the choice of the algorithm, only the ordering of the computations (the schedule) directly influences the volume of communications.We prove lower bounds of $\frac{1}{3\sqrt{2}}\frac{N^3}{\sqrt{S}}$ for the communication volume of the Cholesky factorization of an $N\times N$ symmetric positive definite matrix, and of $\frac{1}{\sqrt{2}}\frac{N^2M}{\sqrt{S}}$ for the SYRK computation of $\mat{A}\cdot\transpose{\mat{A}}$, where $\mathbf{A}$ is an $N\times M$ matrix. Both bounds improve the best known lower bounds from the literature by a factor $\sqrt{2}$.In addition, we present two out-of-core, sequential algorithms with matching communication volume: \TBS for SYRK, with a volume of $\frac{1}{\sqrt{2}}\frac{N^2M}{\sqrt{S}} + \bigo{NM\log N}$, and \LBC for Cholesky, with a volume of $\frac{1}{3\sqrt{2}}\frac{N^3}{\sqrt{S}} + \bigo{N^{5/2}}$. Both algorithms improve over the best known algorithms from the literature by a factor $\sqrt{2}$, and prove that the leading terms in our lower bounds cannot be improved further. This work shows that the operational intensity of symmetric kernels like SYRK or Cholesky is intrinsically higher (by a factor $\sqrt{2}$) than that of corresponding non-symmetric kernels (GEMM and LU factorization).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
