基于BLAS的张量压缩算法的性能预测
On the Performance Prediction of BLAS-based Tensor Contractions
摘要：
张量运算作为各种科学模拟的计算构建块正在激增，并且已知为此类运算开发高性能内核是一项具有挑战性的任务。虽然对于一维和二维张量的操作，存在标准化的接口和高度优化的库（BLAS），但对于更高维的张量，既没有标准也没有高度调优的实现。在本文中，我们考虑任意维数的两个张量之间的收缩，并采取的挑战，通过诉诸序列的BLAS内核生成高性能的实现。该方法在于将收缩分解为仅涉及矩阵或向量的操作。由于一般来说，有许多可供选择的方法来分解一个收缩，我们能够有条不紊地推导出一个大家庭的算法。本文的主要贡献是一个系统的方法，以准确地识别最快的算法在一堆，而不执行它们。相反，这个目标是在底层BLAS内核的一组缓存感知微基准测试的帮助下实现的。我们从这些基准构建的预测使我们能够可靠地挑选出性能最好的算法，而直接执行算法所花费的时间只有一小部分。
英文摘要：
Tensor operations are surging as the computational building blocks for a variety of scientific simulations and the development of high-performance kernels for such operations is known to be a challenging task. While for operations on one- and two-dimensional tensors there exist standardized interfaces and highly-optimized libraries (BLAS), for higher dimensional tensors neither standards nor highly-tuned implementations exist yet. In this paper, we consider contractions between two tensors of arbitrary dimensionality and take on the challenge of generating high-performance implementations by resorting to sequences of BLAS kernels. The approach consists in breaking the contraction down into operations that only involve matrices or vectors. Since in general there are many alternative ways of decomposing a contraction, we are able to methodically derive a large family of algorithms. The main contribution of this paper is a systematic methodology to accurately identify the fastest algorithms in the bunch, without executing them. The goal is instead accomplished with the help of a set of cache-aware micro-benchmarks for the underlying BLAS kernels. The predictions we construct from such benchmarks allow us to reliably single out the best-performing algorithms in a tiny fraction of the time taken by the direct execution of the algorithms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
