DORY：在低成本IoT MCU上自动端到端部署真实世界的DNN
DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost IoT MCUs
摘要：
深度神经网络（DNN）部署在物联网的极端边缘的终端节点上是支持普及的深度学习增强型应用的关键推动因素。基于MCU的低成本终端节点片内存储器有限，通常用暂存器代替高速缓存，以减少面积开销并提高能效--需要在存储层次结构的不同级别之间进行基于DMA的显式存储器传输。在这些系统上映射现代DNN需要积极的拓扑相关的平铺和双缓冲。在这项工作中，我们提出了DORY（面向存储器的部署）-一种自动工具，用于在通常小于1 MB片上SRAM内存的低成本MCU上部署DNN。DORY将平铺抽象为约束编程（CP）问题：它在每个DNN层施加的拓扑约束下最大化L1内存利用率。然后，它生成ANSI C代码来编排片外和片内传输以及计算阶段。此外，为了最大限度地提高速度，DORY增加了CP公式，促进性能有效的瓷砖尺寸。作为DORY的案例研究，我们将GreenWaves Technologies GAP 8作为目标，它是市场上最先进的并行超低功耗MCU级器件之一。在这款器件上，DORY的MAC/周期性能比GreenWaves专有软件解决方案高2.5倍，比在单层STM 32-F746 MCU上的最先进性能高18.1倍。使用我们的工具，GAP-8可以执行1.0 MobileNet-128网络的端到端推断，平均消耗仅63 pJ/MAC@4.3 fps -比STM 32-F746好15.4倍。我们将所有的开发-DORY框架，优化的后端内核和相关的编译器-作为开源软件发布。
英文摘要：
The deployment of Deep Neural Networks (DNNs) on end-nodes at the extreme edge of the Internet-of-Things is a critical enabler to support pervasive Deep Learning-enhanced applications. Low-Cost MCU-based end-nodes have limited on-chip memory and often replace caches with scratchpads, to reduce area overheads and increase energy efficiency -- requiring explicit DMA-based memory transfers between different levels of the memory hierarchy. Mapping modern DNNs on these systems requires aggressive topology-dependent tiling and double-buffering. In this work, we propose DORY (Deployment Oriented to memoRY) - an automatic tool to deploy DNNs on low cost MCUs with typically less than 1MB of on-chip SRAM memory. DORY abstracts tiling as a Constraint Programming (CP) problem: it maximizes L1 memory utilization under the topological constraints imposed by each DNN layer. Then, it generates ANSI C code to orchestrate off- and on-chip transfers and computation phases. Furthermore, to maximize speed, DORY augments the CP formulation with heuristics promoting performance-effective tile sizes. As a case study for DORY, we target GreenWaves Technologies GAP8, one of the most advanced parallel ultra-low power MCU-class devices on the market. On this device, DORY achieves up to 2.5x better MAC/cycle than the GreenWaves proprietary software solution and 18.1x better than the state-of-the-art result on an STM32-F746 MCU on single layers. Using our tool, GAP-8 can perform end-to-end inference of a 1.0-MobileNet-128 network consuming just 63 pJ/MAC on average @ 4.3 fps - 15.4x better than an STM32-F746. We release all our developments - the DORY framework, the optimized backend kernels, and the related heuristics - as open-source software.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
