具有任意界的投射嵌套环的通信最优拼接
Communication-Optimal Tilings for Projective Nested Loops with Arbitrary Bounds
摘要：
减少通信--无论是在存储层次结构的级别之间，还是在网络上的处理器之间--是许多问题的性能优化（在时间和能量方面）的关键组成部分，包括稠密线性代数、粒子交互和机器学习。对于这些问题，这可以表示为嵌套循环计算，以前的基于平铺的方法已经被用来找到执行它们所需的通信的下限和最佳重排或阻塞，以达到这样的下限。然而，这种一般方法通常假设问题的规模很大，这一假设在实践中往往不符合。例如，矩阵乘法的经典$(\text{# arithmetic operations})/(\text{cache size})^{1/2}$下限对于矩阵向量乘法来说并不严格，矩阵向量乘法必须至少读入存储器的$O(\text{# arithmetic operations})$字;机器学习应用中的几乎所有卷积都会出现类似的问题，这些应用使用非常小的过滤器大小（因此，循环边界）。在本文中，我们提供了一种有效的方法来找到和获得，通过一个适当的，有效的可构造的阻塞，通信下界和匹配的平铺实现这些下界的嵌套循环程序与任意循环边界上的多维数组在投影的情况下，其中数组索引的子集的循环索引。我们的方法适用于所有此类问题，无论维度，大小，内存访问模式或数组数量如何，并直接适用于（除其他示例外）矩阵乘法和类似的稠密线性代数运算，张量收缩，n体成对相互作用，逐点卷积和全连接层。
英文摘要：
Reducing communication - either between levels of a memory hierarchy or between processors over a network - is a key component of performance optimization (in both time and energy) for many problems, including dense linear algebra, particle interactions, and machine learning. For these problems, which can be represented as nested-loop computations, previous tiling based approaches have been used to find both lower bounds on the communication required to execute them and optimal rearrangements, or blockings, to attain such lower bounds. However, such general approaches have typically assumed the problem sizes are large, an assumption that is often not met in practice. For instance, the classical $(\text{# arithmetic operations})/(\text{cache size})^{1/2}$ lower bound for matrix multiplication is not tight for matrix-vector multiplications, which must read in at least $O(\text{# arithmetic operations})$ words of memory; similar issues occur for almost all convolutions in machine learning applications, which use extremely small filter sizes (and therefore, loop bounds). In this paper, we provide an efficient way to both find and obtain, via an appropriate, efficiently constructible blocking, communication lower bounds and matching tilings which attain these lower bounds for nested loop programs with arbitrary loop bounds that operate on multidimensional arrays in the projective case, where the array indices are subsets of the loop indices. Our approach works on all such problems, regardless of dimensionality, size, memory access patterns, or number of arrays, and directly applies to (among other examples) matrix multiplication and similar dense linear algebra operations, tensor contractions, n-body pairwise interactions, pointwise convolutions, and fully connected layers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
