arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2105.13331cs.LGeess.SP
收藏

Quantization和在微控制器上部署深度神经网络

Quantization and Deployment of Deep Neural Networks on Microcontrollers

Université Côte d'Azur(蔚蓝海岸大学);CNRS(法国国家科学研究中心);IMT Atlantique(大西洋国立高等工业技术与矿业学校);MILA(蒙特利尔学习算法研究所)

Pierre-Emmanuel Novac, Ghouthi Boukli Hacene, Alain Pegatoquet, Benoît Miramond, Vincent Gripon

摘要:

将人工智能嵌入到低功耗设备上是一项具有挑战性的任务,随着机器学习和硬件设计的最新进展,这项任务已经部分得到了解决。目前,深度神经网络可以部署在嵌入式目标上以执行不同的任务,例如语音识别、目标检测或人类活动识别。然而,深度神经网络在嵌入式设备上的优化仍有空间。这些优化主要解决了功耗、内存和实时限制,同时也简化了边缘部署。此外,仍然需要更好地了解不同用例可以实现什么。这项工作的重点是量化和部署深度神经网络到低功耗32位微控制器。首先概述了在微控制器上嵌入式执行的上下文中相关的量化方法。然后,提出了一种用于端到端深度神经网络训练、量化和部署的新框架。这个框架被称为MicroAI,旨在替代现有的推理引擎(用于微控制器的TensorFlow Lite和STM32 CubeAI)。我们的框架确实可以很容易地调整和/或扩展特定的用例。支持使用单精度的32位浮点以及8位和16位整数上的定点执行。建议的量化方法进行评估与三个不同的数据集(UCI-HAR,口语MNIST和GTSRB)。最后,MicroAI和现有的嵌入式推理引擎之间的比较研究提供了内存和电源效率。使用基于ARM Cortex-M4 F的微控制器(Ambiq Apollo 3和STM32 L452 RE)进行设备上评估。

英文摘要:

Embedding Artificial Intelligence onto low-power devices is a challenging task that has been partly overcome with recent advances in machine learning and hardware design. Presently, deep neural networks can be deployed on embedded targets to perform different tasks such as speech recognition,object detection or Human Activity Recognition. However, there is still room for optimization of deep neural networks onto embedded devices. These optimizations mainly address power consumption,memory and real-time constraints, but also an easier deployment at the edge. Moreover, there is still a need for a better understanding of what can be achieved for different use cases. This work focuses on quantization and deployment of deep neural networks onto low-power 32-bit microcontrollers. The quantization methods, relevant in the context of an embedded execution onto a microcontroller, are first outlined. Then, a new framework for end-to-end deep neural networks training, quantization and deployment is presented. This framework, called MicroAI, is designed as an alternative to existing inference engines (TensorFlow Lite for Microcontrollers and STM32CubeAI). Our framework can indeed be easily adjusted and/or extended for specific use cases. Execution using single precision 32-bit floating-point as well as fixed-point on 8- and 16-bit integers are supported. The proposed quantization method is evaluated with three different datasets (UCI-HAR, Spoken MNIST and GTSRB). Finally, a comparison study between MicroAI and both existing embedded inference engines is provided in terms of memory and power efficiency. On-device evaluation is done using ARM Cortex-M4F-based microcontrollers (Ambiq Apollo3 and STM32L452RE).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑