arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2304.04842cs.LG
收藏

利用 MicroTVM 将机器学习模型部署至边缘端提前编译运行时

Deploying Machine Learning Models to Ahead-of-Time Runtime on Edge Using MicroTVM

Technische Universität Dresden(德累斯顿工业大学);Centre for Tactile Internet with Human-in-the-loop (CeTI)(人机交互触觉互联网中心(CeTI))

Chen Liu, Matthias Jobst, Liyuan Guo, Xinyue Shi, Johannes Partzsch, Christian Mayr

AI总结:

本文提出基于 MicroTVM 的端到端代码生成器,将预训练模型编译为边缘端 C 运行时,并通过 UMA 接口实现算子加速器卸载,在 ARM Cortex M4F 上验证了手势识别应用。

摘要:

在过去几年中,越来越多的 AI 应用被部署到边缘设备上。然而,数据科学家使用 PyTorch 或 TensorFlow 等机器学习框架训练得到的模型,无法在边缘端无缝执行。本文开发了一个端到端代码生成器,利用 MicroTVM(一个面向裸机设备推理的机器学习编译器框架扩展)将预训练模型解析为后端 C 源库。分析表明,特定的计算密集型算子可以通过 Universal Modular Accelerator (UMA) 接口轻松卸载到专用加速器上,而其他算子则在 CPU 核心中处理。通过使用自动生成的提前编译 C 运行时,我们在 ARM Cortex M4F 核心上进行了一个手势识别实验。

英文摘要:

In the past few years, more and more AI applications have been applied to edge devices. However, models trained by data scientists with machine learning frameworks, such as PyTorch or TensorFlow, can not be seamlessly executed on edge. In this paper, we develop an end-to-end code generator parsing a pre-trained model to C source libraries for the backend using MicroTVM, a machine learning compiler framework extension addressing inference on bare metal devices. An analysis shows that specific compute-intensive operators can be easily offloaded to the dedicated accelerator with a Universal Modular Accelerator (UMA) interface, while others are processed in the CPU cores. By using the automatically generated ahead-of-time C runtime, we conduct a hand gesture recognition experiment on an ARM Cortex M4F core.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑