arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.17246cs.LG
收藏

Pex:通过部分执行实现内存高效的微控制器深度学习

Pex: Memory-efficient Microcontroller Deep Learning through Partial Execution

University of Cambridge(剑桥大学);Samsung AI Centre Cambridge(三星剑桥人工智能中心)

Edgar Liberis, Nicholas D. Lane

摘要:

主要由微控制器单元(MCU)驱动的嵌入式和物联网设备可以通过利用设备上的深度学习来变得更加智能。在MCU上进行神经网络推理的主要挑战之一是读写片上存储器(SRAM,< 512 kB)的数量极其有限。SRAM由神经网络层(运算符)输入和输出缓冲器消耗,传统上,这些缓冲器必须在存储器中(具体化)以供运算符执行。我们讨论了一种用于微控制器深度学习的新型执行范例,该范例修改了神经网络的执行,以避免在内存中实现满缓冲区,从而在没有计算开销的情况下大幅减少了SRAM的使用。这是通过利用运营商的属性来实现的,运营商可以一次消耗/产生其输入/输出的一小部分。我们描述了一个部分执行编译器,Pex,它产生内存高效的执行时间表自动识别子图的运营商,其执行可以沿功能(“通道”)维分裂。通过针对内存瓶颈进行结构化修剪,进一步减少内存使用,从而实现网络架构及其执行时间表的协同设计。我们对图像和音频分类模型的评估:(a)为所考虑的任务在低SRAM使用率下建立了最先进的性能,准确度提高了+2.9%;(b)发现通过单独应用部分执行可以减少4倍内存,或者当使用编译器修剪协同设计时可以减少10.5倍内存,同时与以前的工作相比保持分类准确度;(c)使用恢复的SRAM来处理更高分辨率的输入,将视觉唤醒词的准确性提高了+3.9%。

英文摘要:

Embedded and IoT devices, largely powered by microcontroller units (MCUs), could be made more intelligent by leveraging on-device deep learning. One of the main challenges of neural network inference on an MCU is the extremely limited amount of read-write on-chip memory (SRAM, < 512 kB). SRAM is consumed by the neural network layer (operator) input and output buffers, which, traditionally, must be in memory (materialised) for an operator to execute. We discuss a novel execution paradigm for microcontroller deep learning, which modifies the execution of neural networks to avoid materialising full buffers in memory, drastically reducing SRAM usage with no computation overhead. This is achieved by exploiting the properties of operators, which can consume/produce a fraction of their input/output at a time. We describe a partial execution compiler, Pex, which produces memory-efficient execution schedules automatically by identifying subgraphs of operators whose execution can be split along the feature ("channel") dimension. Memory usage is reduced further by targeting memory bottlenecks with structured pruning, leading to the co-design of the network architecture and its execution schedule. Our evaluation of image and audio classification models: (a) establishes state-of-the-art performance in low SRAM usage regimes for considered tasks with up to +2.9% accuracy increase; (b) finds that a 4x memory reduction is possible by applying partial execution alone, or up to 10.5x when using the compiler-pruning co-design, while maintaining the classification accuracy compared to prior work; (c) uses the recovered SRAM to process higher resolution inputs instead, increasing accuracy by up to +3.9% on Visual Wake Words.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑