利用 MicroTVM 将机器学习模型部署至边缘端提前编译运行时
Deploying Machine Learning Models to Ahead-of-Time Runtime on Edge Using MicroTVM
AI总结:
本文提出基于 MicroTVM 的端到端代码生成器,将预训练模型编译为边缘端 C 运行时,并通过 UMA 接口实现算子加速器卸载,在 ARM Cortex M4F 上验证了手势识别应用。
摘要:
在过去几年中,越来越多的 AI 应用被部署到边缘设备上。然而,数据科学家使用 PyTorch 或 TensorFlow 等机器学习框架训练得到的模型,无法在边缘端无缝执行。本文开发了一个端到端代码生成器,利用 MicroTVM(一个面向裸机设备推理的机器学习编译器框架扩展)将预训练模型解析为后端 C 源库。分析表明,特定的计算密集型算子可以通过 Universal Modular Accelerator (UMA) 接口轻松卸载到专用加速器上,而其他算子则在 CPU 核心中处理。通过使用自动生成的提前编译 C 运行时,我们在 ARM Cortex M4F 核心上进行了一个手势识别实验。
英文摘要:
In the past few years, more and more AI applications have been applied to edge devices. However, models trained by data scientists with machine learning frameworks, such as PyTorch or TensorFlow, can not be seamlessly executed on edge. In this paper, we develop an end-to-end code generator parsing a pre-trained model to C source libraries for the backend using MicroTVM, a machine learning compiler framework extension addressing inference on bare metal devices. An analysis shows that specific compute-intensive operators can be easily offloaded to the dedicated accelerator with a Universal Modular Accelerator (UMA) interface, while others are processed in the CPU cores. By using the automatically generated ahead-of-time C runtime, we conduct a hand gesture recognition experiment on an ARM Cortex M4F core.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。