张量收缩的Strassen算法
Strassen's Algorithm for Tensor Contraction
摘要：
张量收缩（TC）是一个重要的计算内核，广泛应用于许多应用。它是矩阵乘法（GEMM）的多维推广。虽然Strassen的GEMM算法在理论和实践上都得到了很好的研究，但将其扩展到加速TC之前还没有进行过。因此，我们相信这是第一篇论文，以证明如何在实践中可以加快张量收缩与Strassen的算法。通过采用一种新的以矩阵为中心的张量布局，即块-分散-矩阵格式，我们可以在概念上将TC视为一般步幅存储的GEMM，并进行隐式张量到矩阵的变换。这一见解使我们能够定制最近的状态的最先进的实现斯特拉森的算法TC，避免显式的换位（排列）和额外的工作空间，并减少内存移动所产生的开销。通过性能模型以及在现代单核、多核和分布式内存并行架构上的实践证明了性能优势，实现了高达1.3倍的加速比。由此产生的实现可以作为各种应用程序的直接替代品，具有显著的加速效果。
英文摘要：
Tensor contraction (TC) is an important computational kernel widely used in numerous applications. It is a multi-dimensional generalization of matrix multiplication (GEMM). While Strassen's algorithm for GEMM is well studied in theory and practice, extending it to accelerate TC has not been previously pursued. Thus, we believe this to be the first paper to demonstrate how one can in practice speed up tensor contraction with Strassen's algorithm. By adopting a Block-Scatter-Matrix format, a novel matrix-centric tensor layout, we can conceptually view TC as GEMM for a general stride storage, with an implicit tensor-to-matrix transformation. This insight enables us to tailor a recent state-of-the-art implementation of Strassen's algorithm to TC, avoiding explicit transpositions (permutations) and extra workspace, and reducing the overhead of memory movement that is incurred. Performance benefits are demonstrated with a performance model as well as in practice on modern single core, multicore, and distributed memory parallel architectures, achieving up to 1.3x speedup. The resulting implementations can serve as a drop-in replacement for various applications with significant speedup.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
