arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.14110cs.LGcs.AIcs.AR
收藏

使用更低位宽累加器实现深度网络更廉价推理

Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators

Technion(以色列理工学院)

Yaniv Blumenfeld, Itay Hubara, Daniel Soudry

AI总结:

针对深度网络中高精度乘积累加造成的硬件瓶颈,本文提出训练与微调方法,使 DNN 可采用廉价的 12 位累加器且准确率基本无损,并以细粒度梯度近似缓解更低精度下的性能下降。

摘要:

关于深度神经网络(DNN)量化的大多数研究都集中于降低高层框架可见张量(例如权重、激活和梯度)的精度。然而,当前硬件仍依赖高精度核心运算,其中最重要的是乘积累加运算。这种高精度累加运算正逐渐成为主要计算瓶颈,原因在于迄今为止,使用低精度累加器会导致性能显著下降。在本工作中,我们提出一种简单方法来训练和微调高端 DNN,首次使其能够使用更廉价的 12 位累加器,而准确率没有显著下降。最后,我们表明,当进一步降低累加精度时,使用细粒度梯度近似可以提升 DNN 准确率。

英文摘要:

The majority of the research on the quantization of Deep Neural Networks (DNNs) is focused on reducing the precision of tensors visible by high-level frameworks (e.g., weights, activations, and gradients). However, current hardware still relies on high-accuracy core operations. Most significant is the operation of accumulating products. This high-precision accumulation operation is gradually becoming the main computational bottleneck. This is because, so far, the usage of low-precision accumulators led to a significant degradation in performance. In this work, we present a simple method to train and fine-tune high-end DNNs, to allow, for the first time, utilization of cheaper, $12$-bits accumulators, with no significant degradation in accuracy. Lastly, we show that as we decrease the accumulation precision further, using fine-grained gradient approximations can improve the DNN accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑