基于约束编程的联合程序和布局变换在专用硬件上实现卷积运算
Joint Program and Layout Transformations to enable Convolutional Operators on Specialized Hardware based on Constraint Programming
摘要：
深度人工神经网络（DNN）在许多领域的成功，为计算密集型DNN操作员的硬件加速器创造了丰富的研究。然而，用复杂的硬件内在函数（如矩阵乘法）有效地实现这些运算符是一项尚未优雅地自动化的任务。解决这个任务通常需要联合程序和数据布局转换。已经提出了这个问题的第一个解决方案，例如TVM，UNIT或ISAMIR，它们在运算符的循环级表示上工作，并在执行嵌入到运算符之前指定数据布局和可能的程序转换。这种自上而下的方法在探索范围和搜索空间复杂性之间产生了紧张关系，特别是在还探索数据布局转换（如im2col、通道打包或填充）时。在这项工作中，我们提出了一个新的方法来解决这个问题。我们创建了一个自下而上的方法，允许基于发现的嵌入的计算和数据布局的联合转换。通过将嵌入公式化为标量多项式上的约束满足问题，每个可能的嵌入解都包含在搜索空间中。向求解器添加附加约束和优化目标会生成首选解决方案的子集。使用VTA硬件加速器与百度DeepBench推理基准测试的评估表明，我们的方法可以自动生成与参考实现竞争的代码。此外，我们表明，动态确定的数据布局的基础上的内在和工作负载是有益的硬件利用率和性能。在参考实现由于其固定部署策略而具有低硬件利用率的情况下，我们实现了高达x2.813的几何加速比，而单个运营商可以提高多达x170。
英文摘要：
The success of Deep Artificial Neural Networks (DNNs) in many domains created a rich body of research concerned with hardware accelerators for compute-intensive DNN operators. However, implementing such operators efficiently with complex hardware intrinsics such as matrix multiply is a task not yet automated gracefully. Solving this task often requires joint program and data layout transformations. First solutions to this problem have been proposed, such as TVM, UNIT or ISAMIR, which work on a loop-level representation of operators and specify data layout and possible program transformations before the embedding into the operator is performed. This top-down approach creates a tension between exploration range and search space complexity, especially when also exploring data layout transformations such as im2col, channel packing or padding. In this work, we propose a new approach to this problem. We created a bottom-up method that allows the joint transformation of both compuation and data layout based on the found embedding. By formulating the embedding as a constraint satisfaction problem over the scalar dataflow, every possible embedding solution is contained in the search space. Adding additional constraints and optmization targets to the solver generates the subset of preferable solutions. An evaluation using the VTA hardware accelerator with the Baidu DeepBench inference benchmark shows that our approach can automatically generate code competitive to reference implementations. Further, we show that dynamically determining the data layout based on intrinsic and workload is beneficial for hardware utilization and performance. In cases where the reference implementation has low hardware utilization due to its fixed deployment strategy, we achieve a geomean speedup of up to x2.813, while individual operators can improve as much as x170.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
