TensorFlow Lite 中动态可重构可变精度稀疏-稠密矩阵加速
Dynamically Reconfigurable Variable-precision Sparse-Dense Matrix Acceleration in Tensorflow Lite
AI总结:
本文提出动态可重构加速器 FADES,集成于 TensorFlow Lite,通过 int8/float 精度切换在资源受限 SoC 上实现稀疏与稠密矩阵加速,性能优于脉动阵列并较 NEON RUY 库最高加速 20 倍。
摘要:
本文提出了一种名为 FADES(稠密与稀疏矩阵融合架构)的动态可重构硬件加速器。FADES 设计提供多种配置选项,利用数据流模型在并行度与复杂度之间进行权衡,形成读取、计算、缩放和写回结果的四个阶段。FADES 被映射到可编程逻辑(PL),并与运行在异构 SoC 器件处理系统(PS)上的 TensorFlow Lite 推理引擎集成。该加速器用于计算张量运算,而动态可重构方法可用于在 int8 和 float 模式之间切换精度。与同时支持两种算术精度相比,这种动态重构通过允许更多核心映射到资源受限器件上实现更高性能,并降低功耗。我们将所提出的硬件与面向稠密矩阵的高性能脉动阵列架构进行比较,在相同工艺下以一半的 DSP 模块在稠密模式下获得 25% 的性能提升。在稀疏模式下,我们证明即使在低稀疏度水平下,该核心也能超越稠密模式,并且单核心相比软件优化的 NEON RUY 库实现最高 20 倍加速。
英文摘要:
In this paper, we present a dynamically reconfigurable hardware accelerator called FADES (Fused Architecture for DEnse and Sparse matrices). The FADES design offers multiple configuration options that trade off parallelism and complexity using a dataflow model to create four stages that read, compute, scale and write results. FADES is mapped to the programmable logic (PL) and integrated with the TensorFlow Lite inference engine running on the processing system (PS) of a heterogeneous SoC device. The accelerator is used to compute the tensor operations, while the dynamically reconfigurable approach can be used to switch precision between int8 and float modes. This dynamic reconfiguration enables better performance by allowing more cores to be mapped to the resource-constrained device and lower power consumption compared with supporting both arithmetic precisions simultaneously. We compare the proposed hardware with a high-performance systolic architecture for dense matrices obtaining 25% better performance in dense mode with half the DSP blocks in the same technology. In sparse mode, we show that the core can outperform dense mode even at low sparsity levels, and a single-core achieves up to 20x acceleration over the software-optimized NEON RUY library.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。