面向多核 NPU 的张量切片与优化
Tensor Slicing and Optimization for Multicore NPUs
AI总结：
本文提出用于多核NPU的TensorFlow XLA/LLVM编译器优化过程TSO，通过最大化跨核心卷积并行度与利用DRAM突发时间估计指导张量切片减少数据传输，在32核NMP上实现最高21.7%的加速，并验证了其在Glow框架上的通用性。
摘要：
尽管卷积神经网络（CNN）模型的代码生成已被广泛研究，但在高度受限的多核神经处理器单元（NPU）上执行高效的数据切片和并行化仍然是一个具有挑战性的问题。鉴于卷积输入/输出张量的尺寸以及 NPU 片上存储器的小容量，在最大化并行度和 MAC 利用率的同时最小化内存事务，是任何有效解决方案的核心。本文提出了一种用于多核 NPU 的 TensorFlow XLA/LLVM 编译器优化过程，称为张量切片优化（TSO），它：(a) 最大化跨 NPU 核心的卷积并行度和内存使用率；(b) 通过使用 DRAM 内存突发时间估计来指导张量切片，减少主机和 NPU 片上存储器之间的数据传输。为了评估所提出的方法，使用神经形态处理器（NMP）进行了一组实验，这是一个包含 32 个 RISC-V 核心并扩展了新型 CNN 指令的多核 NPU。实验结果表明，TSO 能够识别出最佳的张量切片，从而最小化一组 CNN 模型的执行时间。将 TSO 基于突发的技术与无突发数据切片方法进行比较时，速度提升了高达 21.7%。为了验证 TSO 方法的通用性，该算法也被移植到 Glow 机器学习框架中。这些模型的性能在 Glow 和 TensorFlow XLA/LLVM 编译器上均进行了测量，结果显示出相似的结果。
英文摘要：
Although code generation for Convolution Neural Network (CNN) models has been extensively studied, performing efficient data slicing and parallelization for highly-constrai\-ned Multicore Neural Processor Units (NPUs) is still a challenging problem. Given the size of convolutions' input/output tensors and the small footprint of NPU on-chip memories, minimizing memory transactions while maximizing parallelism and MAC utilization are central to any effective solution. This paper proposes a TensorFlow XLA/LLVM compiler optimization pass for Multicore NPUs, called Tensor Slicing Optimization (TSO), which: (a) maximizes convolution parallelism and memory usage across NPU cores; and (b) reduces data transfers between host and NPU on-chip memories by using DRAM memory burst time estimates to guide tensor slicing. To evaluate the proposed approach, a set of experiments was performed using the NeuroMorphic Processor (NMP), a multicore NPU containing 32 RISC-V cores extended with novel CNN instructions. Experimental results show that TSO is capable of identifying the best tensor slicing that minimizes execution time for a set of CNN models. Speed-ups of up to 21.7\% result when comparing the TSO burst-based technique to a no-burst data slicing approach. To validate the generality of the TSO approach, the algorithm was also ported to the Glow Machine Learning framework. The performance of the models were measured on both Glow and TensorFlow XLA/LLVM compilers, revealing similar results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
