面向深度学习推理中比特稀疏性利用的跳零MAC设计的商业评估
Commercial Evaluation of Zero-Skipping MAC Design for Bit Sparsity Exploitation in DL Inference
AI总结:
本文提出OzMAC跳零MAC设计,通过TSMC N5工艺综合后评估证明其在多个位宽和频率下相比二进制MAC显著改善面积、功耗和能量,验证了利用比特稀疏性的商业实现潜力。
摘要:
通用矩阵乘法(GEMM)单元由乘累加(MAC)阵列组成,承担了深度学习(DL)中的大部分计算。近期工作提出了一种新颖的MAC设计,称为Bit-Pragmatic(PRA),能够动态利用比特稀疏性。本文提出了OzMAC(Omit-zero-MAC),这是对PRA的修改后重新实现,但超越了先前的工作,通过使用TSMC N5工艺节点,在多个位宽和时钟频率下对二进制MAC设计进行严格的综合后评估,以评估其商业实现潜力。我们证明了八个预训练的INT8深度学习工作负载中存在高比特稀疏性,并表明8位OzMAC在面积、功耗和能量三个指标上分别显著改善了21%、70%和28%。当扩展数据精度(4、8、16位)和时钟频率(0.5 GHz、1 GHz、1.5 GHz)时,也取得了类似的改进。对于8位OzMAC,通过扩展其频率以归一化吞吐量,它仍然在功耗和能量上均实现了30%的改善。
英文摘要:
General Matrix Multiply (GEMM) units, consisting of multiply-accumulate (MAC) arrays, perform bulk of the computation in deep learning (DL). Recent work has proposed a novel MAC design, Bit-Pragmatic (PRA), capable of dynamically exploiting bit sparsity. This work presents OzMAC (Omit-zero-MAC), a modified re-implementation of PRA, but extends beyond earlier works by performing rigorous post-synthesis evaluation against binary MAC design across multiple bitwidths and clock frequencies using TSMC N5 process node to assess commercial implementation potential. We demonstrate the existence of high bit sparsity in eight pretrained INT8 DL workloads and show that 8-bit OzMAC improves all three metrics of area, power, and energy significantly by 21%, 70%, and 28%, respectively. Similar improvements are achieved when scaling data precisions (4, 8, 16 bits) and clock frequencies (0.5 GHz, 1 GHz, 1.5 GHz). For the 8-bit OzMAC, scaling its frequency to normalize the throughput, it still achieves 30% improvement on both power and energy.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。