UNIT：统一张量指令编译
UNIT: Unifying Tensorized Instruction Compilation
摘要：
由于DNN对计算的需求不断增加，研究人员开发了硬件和软件机制来减少计算和内存负担。一种广泛采用的方法是使用混合精度数据类型。然而，由于数据转换的开销，在没有硬件支持的情况下很难利用混合精度。硬件供应商提供用于混合精度张量运算的张量化指令，如Intel VNNI、Tensor Core和ARM-DOT。这些指令涉及一种计算习惯用法，它将多个低精度元素减少为一个高精度元素。缺乏编译技术使得很难利用这些指令：使用供应商提供的计算密集型内核库是不灵活的，并阻止进一步优化，手动编写硬件intrinsic容易出错，对程序员来说很困难。一些先前的作品通过为每个指令创建编译器来解决这个问题。当涉及到许多张量化指令时，这需要付出过多的努力。在这项工作中，我们开发了一个编译器框架来统一这些指令的编译-统一的语义抽象简化了新指令的集成，并重用分析和转换。来自不同平台的Tensorized指令可以通过UNIT进行编译，并具有良好的性能。给定一个张量化指令和一个张量操作，UNIT会自动检测适用性，转换操作的循环组织，并重写循环体以利用张量化指令。根据我们的评估，UNIT可以针对各种主流硬件平台。生成的端到端推理模型在x86 CPU上实现了Intel oneDNN的1.3倍加速，在NvidiaGPU上实现了Nvidia cuDNN的1.75倍加速，在ARM CPU上实现了精心调整的ARM DOT TVM解决方案的1.13倍加速。
英文摘要：
Because of the increasing demand for computation in DNN, researchers develope both hardware and software mechanisms to reduce the compute and memory burden. A widely adopted approach is to use mixed precision data types. However, it is hard to leverage mixed precision without hardware support because of the overhead of data casting. Hardware vendors offer tensorized instructions for mixed-precision tensor operations, like Intel VNNI, Tensor Core, and ARM-DOT. These instructions involve a computing idiom that reduces multiple low precision elements into one high precision element. The lack of compilation techniques for this makes it hard to utilize these instructions: Using vendor-provided libraries for computationally-intensive kernels is inflexible and prevents further optimizations, and manually writing hardware intrinsics is error-prone and difficult for programmers. Some prior works address this problem by creating compilers for each instruction. This requires excessive effort when it comes to many tensorized instructions. In this work, we develop a compiler framework to unify the compilation for these instructions -- a unified semantics abstraction eases the integration of new instructions, and reuses the analysis and transformations. Tensorized instructions from different platforms can be compiled via UNIT with moderate effort for favorable performance. Given a tensorized instruction and a tensor operation, UNIT automatically detects the applicability, transforms the loop organization of the operation,and rewrites the loop body to leverage the tensorized instruction. According to our evaluation, UNIT can target various mainstream hardware platforms. The generated end-to-end inference model achieves 1.3x speedup over Intel oneDNN on an x86 CPU, 1.75x speedup over Nvidia cuDNN on an NvidiaGPU, and 1.13x speedup over a carefully tuned TVM solution for ARM DOT on an ARM CPU.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
