34 pages, 14 figures. Published in MDPI Sensors 2021, special issue "Embedded Artificial Intelligence (AI) for Smart Sensing and IoT Applications": https://www.mdpi.com/1424-8220/21/9/2984 . v2: add reference for MicroAI software and link to source code repository; fix Eq. 3 according to implementation; improve English grammar and spelling; improve page layout
将人工智能嵌入到低功耗设备上是一项具有挑战性的任务,随着机器学习和硬件设计的最新进展,这项任务已经部分得到了解决。目前,深度神经网络可以部署在嵌入式目标上以执行不同的任务,例如语音识别、目标检测或人类活动识别。然而,深度神经网络在嵌入式设备上的优化仍有空间。这些优化主要解决了功耗、内存和实时限制,同时也简化了边缘部署。此外,仍然需要更好地了解不同用例可以实现什么。这项工作的重点是量化和部署深度神经网络到低功耗32位微控制器。首先概述了在微控制器上嵌入式执行的上下文中相关的量化方法。然后,提出了一种用于端到端深度神经网络训练、量化和部署的新框架。这个框架被称为MicroAI,旨在替代现有的推理引擎(用于微控制器的TensorFlow Lite和STM32 CubeAI)。我们的框架确实可以很容易地调整和/或扩展特定的用例。支持使用单精度的32位浮点以及8位和16位整数上的定点执行。建议的量化方法进行评估与三个不同的数据集(UCI-HAR,口语MNIST和GTSRB)。最后,MicroAI和现有的嵌入式推理引擎之间的比较研究提供了内存和电源效率。使用基于ARM Cortex-M4 F的微控制器(Ambiq Apollo 3和STM32 L452 RE)进行设备上评估。
英文摘要:
Embedding Artificial Intelligence onto low-power devices is a challenging task that has been partly overcome with recent advances in machine learning and hardware design. Presently, deep neural networks can be deployed on embedded targets to perform different tasks such as speech recognition,object detection or Human Activity Recognition. However, there is still room for optimization of deep neural networks onto embedded devices. These optimizations mainly address power consumption,memory and real-time constraints, but also an easier deployment at the edge. Moreover, there is still a need for a better understanding of what can be achieved for different use cases. This work focuses on quantization and deployment of deep neural networks onto low-power 32-bit microcontrollers. The quantization methods, relevant in the context of an embedded execution onto a microcontroller, are first outlined. Then, a new framework for end-to-end deep neural networks training, quantization and deployment is presented. This framework, called MicroAI, is designed as an alternative to existing inference engines (TensorFlow Lite for Microcontrollers and STM32CubeAI). Our framework can indeed be easily adjusted and/or extended for specific use cases. Execution using single precision 32-bit floating-point as well as fixed-point on 8- and 16-bit integers are supported. The proposed quantization method is evaluated with three different datasets (UCI-HAR, Spoken MNIST and GTSRB). Finally, a comparison study between MicroAI and both existing embedded inference engines is provided in terms of memory and power efficiency. On-device evaluation is done using ARM Cortex-M4F-based microcontrollers (Ambiq Apollo3 and STM32L452RE).
FEMBA on the Edge: Physiologically-Aware Pre-Training, Quantization, and Deployment of a Bidirectional Mamba EEG Foundation Model on an Ultra-low Power Microcontroller
Integrated Systems Laboratory, ETH Zürich(苏黎世联邦理工学院集成系统实验室);DEI, University of Bologna(博洛尼亚大学电气、电子与信息工程系)
Real-Time Stress Detection via Photoplethysmogram Signals: Implementation of a Combined Continuous Wavelet Transform and Convolutional Neural Network on Resource-Constrained Microcontrollers
University of Tehran(德黑兰大学);Kettering University(凯特林大学)
Value-Driven Mixed-Precision Quantization for Patch-Based Inference on Microcontrollers
Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology(华中科技大学武汉光电国家研究中心);Ping An Technology (Shenzhen) Co., Ltd.(平安科技(深圳)有限公司)
From Bits to Chips: An LLM-based Hardware-Aware Quantization Agent for Streamlined Deployment of LLMs
The University of Arizona(亚利桑那大学);Clemson University(克莱姆森大学);University of Georgia(佐治亚大学);The University of Tulsa(塔尔萨大学);Northeastern University(东北大学);Western Digital Corporation(西部数据公司)
Technical Report: NEMO DNN Quantization for Deployment Model
IIS.EE.ETHZ(苏黎世联邦理工学院电子工程研究所)
本技术报告旨在为深度神经网络(DNN)分层量化定义一个正式框架,特别关注与最终部署相关的问题。它也是NEMO(NEural Minimization for pytOrch)框架的文档。它描述了NEMO中使用的四种DNN表示(FullPrecision,FakeQuantized,QuantizedDeployable和IntegerDeployable),特别关注后两者的正式定义。该模型的一个重要特征,特别是IntegerDeployable表示,是它能够使用纯整数进行DNN推理,而无需在计算的任何部分诉诸实数,也无需依赖显式的定点数值表示。
PolyChirp: Multi-Species Birdsong Classification Using TinyML on Low-Power Acoustic Sensors
École Polytechnique(巴黎综合理工学院);Freie Universität Berlin(柏林自由大学);Berlin University of Applied Sciences (BHT)(柏林应用科技大学);HES-SO Valais-Wallis(瓦莱州瓦利斯高等专业学院);Inria(法国国家信息与自动化研究所)
AutoMCU: Feasibility-First MCU Neural Network Customization via LLM-based Multi-Agent Systems
School of Computing and Artificial Intelligence, Southwest Jiaotong University(计算机与人工智能学院,西南交通大学);Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China(深圳先进研究院,电子科技大学);School of Computer Science and Engineering, Northeastern University(计算机科学与工程学院,东北大学)
Efficient Deployment of Spiking Neural Networks on SpiNNaker2 for DVS Gesture Recognition Using Neuromorphic Intermediate Representation
Technische Universität Dresden(德累斯顿工业大学);ScaDS.AI Dresden/Leipzig(德累斯顿/莱佩斯人工智能服务中心);Centre for Tactile Internet with Human-in-the-Loop (CeTI)(以人为中心的触觉互联网中心)
NeuroEdge: Real-Time Hand Gesture Recognition with High-Density EMG Using Deep Learning at the Edge
San Francisco State University(旧金山州立大学);Department of Biology(生物系);School of Engineering in Computer Engineering(计算机工程学院);College of San Mateo(圣马特奥学院);Contra Costa College(康特拉科斯塔学院)