arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.02473cs.ARcs.LG
收藏

迈向基于注意力的 TinyML:一种异构加速架构与自动化部署流程

Toward Attention-based TinyML: A Heterogeneous Accelerated Architecture and Automated Deployment Flow

Philip Wiese, Gamze İslamoğlu, Moritz Scherer, Luka Macan, Victor J. B. Jung, Alessio Burrello, Francesco Conti, Luca Benini

AI总结:

针对 tinyML 难以支持 Transformer 的问题,本文提出 RISC-V 八核集群与量化 Attention 硬连线加速器耦合的异构架构及自动化部署流程,实现端到端 8 位推理并取得领先能效与吞吐量。

摘要:

Tiny Machine Learning(tinyML,微型机器学习)面临的挑战之一,是跟上机器学习模型从卷积神经网络到 Transformers 的演进。我们通过利用一种异构架构模板来解决该问题,该模板将 RISC-V 处理器与硬连线加速器耦合,并由自动化部署流程提供支持。我们在 tinyML 功耗范围内展示了基于注意力的模型,采用一个八核集群并耦合用于量化 Attention 的加速器。我们的部署流程支持端到端 8 位 Transformer 推理,实现了前沿的能效和吞吐量:2960 GOp/J 与 154 GOp/s(0.65 V、22 nm FD-SOI 工艺)。

英文摘要:

One of the challenges for Tiny Machine Learning (tinyML) is keeping up with the evolution of Machine Learning models from Convolutional Neural Networks to Transformers. We address this by leveraging a heterogeneous architectural template coupling RISC-V processors with hardwired accelerators supported by an automated deployment flow. We demonstrate Attention-based models in a tinyML power envelope with an octa-core cluster coupled with an accelerator for quantized Attention. Our deployment flow enables end-to-end 8-bit Transformer inference, achieving leading-edge energy efficiency and throughput of 2960 GOp/J and 154 GOp/s (0.65 V, 22 nm FD-SOI technology).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑