使用卷积切片优化和ISA扩展推进直接卷积
Advancing Direct Convolution using Convolution Slicing Optimization and ISA Extensions
摘要：
卷积是机器学习模型推理必须执行的计算最密集的操作之一。计算卷积的传统方法称为Im 2Col + BLAS方法。本文提出SConv：基于MLIR/LLVM代码生成工具链的直接卷积算法，可以集成到机器学习编译器中。该算法引入：（b）卷积切片优化（CSO）-使用CSA生成平铺直接卷积宏内核的代码生成通道;以及（c）基于向量的打包（VBP）-基于向量的架构特定的优化输入张量打包解决方案-寄存器移位指令，用于具有单位步幅的卷积。在来自完整ONNX-MLIR机器学习模型的393个卷积上进行的实验表明，消除Im 2Col转换和使用快速打包例程会导致完整模型推理的总打包时间减少，在Intel x86上为2.0 x-3.9 x，在IBM POWER 10上为3.6 x-7.2 x。基于当前BLAS实现的Im 2Col + BLAS方法的端到端机器学习模型推理的速度提升对于Intel x86在9% - 25%的范围内，对于IBM POWER 10架构在10% - 42%的范围内。模型推理的总卷积加速在Intel x86上为12% - 27%，在IBM POWER 10上为26% - 46%。SConv在计算逐点卷积时，在219个测试实例中的83%以上，也优于BLAS GEMM。
英文摘要：
Convolution is one of the most computationally intensive operations that must be performed for machine-learning model inference. A traditional approach to compute convolutions is known as the Im2Col + BLAS method. This paper proposes SConv: a direct-convolution algorithm based on a MLIR/LLVM code-generation toolchain that can be integrated into machine-learning compilers . This algorithm introduces: (a) Convolution Slicing Analysis (CSA) - a convolution-specific 3D cache-blocking analysis pass that focuses on tile reuse over the cache hierarchy; (b) Convolution Slicing Optimization (CSO) - a code-generation pass that uses CSA to generate a tiled direct-convolution macro-kernel; and (c) Vector-Based Packing (VBP) - an architecture-specific optimized input-tensor packing solution based on vector-register shift instructions for convolutions with unitary stride. Experiments conducted on 393 convolutions from full ONNX-MLIR machine-learning models indicate that the elimination of the Im2Col transformation and the use of fast packing routines result in a total packing time reduction, on full model inference, of 2.0x - 3.9x on Intel x86 and 3.6x - 7.2x on IBM POWER10. The speed-up over an Im2Col + BLAS method based on current BLAS implementations for end-to-end machine-learning model inference is in the range of 9% - 25% for Intel x86 and 10% - 42% for IBM POWER10 architectures. The total convolution speedup for model inference is 12% - 27% on Intel x86 and 26% - 46% on IBM POWER10. SConv also outperforms BLAS GEMM, when computing pointwise convolutions, in more than 83% of the 219 tested instances.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
