MATCH：面向异构边缘设备的模型感知基于TVM的编译
MATCH: Model-Aware TVM-based Compilation for Heterogeneous Edge Devices
AI总结：
针对异构边缘设备DNN部署中定制工具链难移植与通用工具链难优化的痛点，提出基于TVM的MATCH框架，通过硬件成本模型实现敏捷重定向并超越定制工具链性能。
摘要：
在异构边缘平台上简化深度神经网络（DNN）的部署，即将同一微控制器单元（MCU）内的指令处理器和用于张量计算的硬件加速器耦合起来，正成为TinyML领域的关键挑战之一。性能最佳的DNN编译工具链通常针对单一MCU系列进行深度定制，而移植到不同的异构MCU系列意味着需要对几乎整个编译器进行耗费人力的重新开发。相反，诸如TVM等可重定向的工具链无法利用定制加速器的能力，导致生成通用但未优化的代码。为了克服这种二元对立，我们引入了MATCH，这是一种新颖的基于TVM的DNN部署框架，得益于可定制的基于模型的硬件抽象，它专为在不同MCU处理器和加速器之间实现轻松敏捷的重定向而设计。我们表明，通过硬件成本模型增强的通用且可重定向的映射框架，能够在不同目标上与定制工具链竞争甚至超越它们，同时仅需定义抽象硬件模型和特定于SoC的API。我们在两款最先进的异构MCU（GAP9和DIANA）上测试了MATCH。在MLPerf Tiny基准套件的四个DNN模型上，得益于对板载硬件加速器的利用，与使用普通TVM相比，MATCH在DIANA上将推理延迟最多降低了60.88倍。与完全针对DIANA定制的工具链HTVM相比，我们仍将延迟降低了16.94%。在GAP9上，使用相同的基准测试，得益于我们的异构DNN映射方法协同利用了DNN加速器和板载的八核集群，与专用的DORY编译器相比，我们将延迟改善了2.15倍。
英文摘要：
Streamlining the deployment of Deep Neural Networks (DNNs) on heterogeneous edge platforms, coupling within the same micro-controller unit (MCU) instruction processors and hardware accelerators for tensor computations, is becoming one of the crucial challenges of the TinyML field. The best-performing DNN compilation toolchains are usually deeply customized for a single MCU family, and porting to a different heterogeneous MCU family implies labor-intensive re-development of almost the entire compiler. On the opposite side, retargetable toolchains, such as TVM, fail to exploit the capabilities of custom accelerators, resulting in the generation of general but unoptimized code. To overcome this duality, we introduce MATCH, a novel TVM-based DNN deployment framework designed for easy agile retargeting across different MCU processors and accelerators, thanks to a customizable model-based hardware abstraction. We show that a general and retargetable mapping framework enhanced with hardware cost models can compete with and even outperform custom toolchains on diverse targets while only needing the definition of an abstract hardware model and a SoC-specific API. We tested MATCH on two state-of-the-art heterogeneous MCUs, GAP9 and DIANA. On the four DNN models of the MLPerf Tiny suite MATCH reduces inference latency by up to 60.88 times on DIANA, compared to using the plain TVM, thanks to the exploitation of the on-board HW accelerator. Compared to HTVM, a fully customized toolchain for DIANA, we still reduce the latency by 16.94%. On GAP9, using the same benchmarks, we improve the latency by 2.15 times compared to the dedicated DORY compiler, thanks to our heterogeneous DNN mapping approach that synergically exploits the DNN accelerator and the eight-cores cluster available on board.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
