arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2207.03192cs.AR
收藏

MiniFloat-NN 与 ExSdotp:面向 RISC-V 核低精度训练的 ISA 扩展与模块化开放硬件单元

MiniFloat-NN and ExSdotp: An ISA Extension and a Modular Open Hardware Unit for Low-Precision Training on RISC-V cores

Luca Bertaccini, Gianna Paulin, Tim Fischer, Stefan Mach, Luca Benini

AI总结:

本文提出 RISC-V 的 MiniFloat-NN 低精度训练 ISA 扩展及融合 ExSdotp 硬件单元,支持多种 FP8/FP16 运算,并在 12 nm 八核集群中实现 575 GFLOPS/W。

摘要:

低精度格式近来通过降低神经网络(NN)模型的内存占用并提升底层硬件架构的能效,推动了神经网络训练和推理的重大突破。窄整数数据类型已在 NN 推理中得到广泛研究,并被成功推进到三值和二值表示的极端。相比之下,大多数面向训练的平台至少使用 16 位浮点(FP)格式。8 位 FP 格式等更低精度数据类型以及混合精度技术直到最近才在硬件实现中得到探索。我们提出 MiniFloat-NN,这是一种用于低精度 NN 训练的 RISC-V 指令集架构扩展,支持两种 8 位和两种 16 位 FP 格式以及扩展运算。该扩展包括以更大格式累加结果的点积之和指令,以及采用扩展和非扩展两种变体的三项加法。我们实现了一个 ExSdotp 单元,以在硬件中高效支持这两类指令。ExSdotp 模块的融合特性避免了两次连续 FP 加法的非结合性所产生的精度损失,同时与级联两个扩展型融合乘加单元相比,节省约 30% 的面积并缩短关键路径。我们将 ExSdotp 模块复制到 SIMD 封装器中,并将其集成到一个开源浮点单元;该单元与开源 RISC-V 核耦合,为未来面向低精度和混合精度 NN 训练的可扩展架构奠定基础。一个包含八个扩展核且共享便签式存储器的集群采用 12 nm FinFET 工艺实现,在 0.8 V、1.26 GHz 下计算 FP8 到 FP16 的 GEMM 时,实现高达 575 GFLOPS/W 的能效。

英文摘要:

Low-precision formats have recently driven major breakthroughs in neural network (NN) training and inference by reducing the memory footprint of the NN models and improving the energy efficiency of the underlying hardware architectures. Narrow integer data types have been vastly investigated for NN inference and have successfully been pushed to the extreme of ternary and binary representations. In contrast, most training-oriented platforms use at least 16-bit floating-point (FP) formats. Lower-precision data types such as 8-bit FP formats and mixed-precision techniques have only recently been explored in hardware implementations. We present MiniFloat-NN, a RISC-V instruction set architecture extension for low-precision NN training, providing support for two 8-bit and two 16-bit FP formats and expanding operations. The extension includes sum-of-dot-product instructions that accumulate the result in a larger format and three-term additions in two variations: expanding and non-expanding. We implement an ExSdotp unit to efficiently support in hardware both instruction types. The fused nature of the ExSdotp module prevents precision losses generated by the non-associativity of two consecutive FP additions while saving around 30% of the area and critical path compared to a cascade of two expanding fused multiply-add units. We replicate the ExSdotp module in a SIMD wrapper and integrate it into an open-source floating-point unit, which, coupled to an open-source RISC-V core, lays the foundation for future scalable architectures targeting low-precision and mixed-precision NN training. A cluster containing eight extended cores sharing a scratchpad memory, implemented in 12 nm FinFET technology, achieves up to 575 GFLOPS/W when computing FP8-to-FP16 GEMMs at 0.8 V, 1.26 GHz.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑