arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.18953cs.AR
收藏

StruM:面向高效深度学习硬件协同设计的结构化混合精度方法

StruM: Structured Mixed Precision for Efficient Deep Learning Hardware Codesign

Michael Wu, Arnab Raha, Deepak A. Mathaikutty, Martin Langhammer, Engin Tunali, Daksha Sharma

AI总结:

本研究提出与硬件加速器协同设计的结构化混合精度推理方法StruM,无需重训练即可将8位权值量化为4位且精度损失极小,在自研加速器上实现了功耗与面积的显著降低。

摘要:

本文提出StruM,这是一种新型的基于结构化混合精度的深度学习推理方法,与其配套的硬件加速器(DPU,数据处理单元)协同设计,旨在解决数据中心和边缘应用中深度学习工作负载不断攀升的计算与内存需求问题。与传统方法不同,我们的方法无需耗时的重新训练/微调,也无需专用硬件访问权限。通过利用层内权值幅值的方差,我们将块内的权值量化为两种不同精度等级,在各类卷积神经网络(CNNs)中,可将8位整型权值的精度最高降低50%至4位,且推理精度损失可忽略不计。为验证利用混合精度带来的效率提升,我们在自研的支持低精度与混合精度执行的FlexNN DNN加速器[1]之上实现了StruM。实验结果表明,所提出的基于StruM的硬件架构在处理单元(PE)层面实现了31-34%的功耗降低,在加速器层面实现了10%的面积减少。此外,静态配置的StruM在PE层面可减少23-26%的面积,在DPU层面可节省2-3%的面积。

英文摘要:

In this paper, we propose StruM, a novel structured mixed-precision-based deep learning inference method, co-designed with its associated hardware accelerator (DPU), to address the escalating computational and memory demands of deep learning workloads in data centers and edge applications. Diverging from traditional approaches, our method avoids time-consuming re-training/fine-tuning and specialized hardware access. By leveraging the variance in weight magnitudes within layers, we quantize values within blocks to two different levels, achieving up to a 50% reduction in precision for 8-bit integer weights to 4-bit values across various Convolutional Neural Networks (CNNs) with negligible loss in inference accuracy. To demonstrate efficiency gains by utilizing mixed precision, we implement StruM on top of our in-house FlexNN DNN accelerator [1] that supports low and mixed-precision execution. Experimental results depict that the proposed StruM-based hardware architecture achieves a 31-34% reduction in processing element (PE) power consumption and a 10% reduction in area at the accelerator level. In addition, the statically configured StruM results in 23-26% area reduction at the PE level and 2-3% area savings at the DPU level.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑