AdaptivFloat：一种用于弹性深度学习推理的基于浮点的数据类型
AdaptivFloat: A Floating-point based Data Type for Resilient Deep Learning Inference
摘要：
传统的硬件友好的量化方法，如定点或整数，往往在非常低的字长表现不佳，因为它们缩小的动态范围不能充分捕获序列转导模型中常见的宽数据分布。我们提出了AdaptivFloat，这是一种用于深度学习的浮点启发数字表示格式，可以在层粒度上动态地最大化和最佳地裁剪其可用动态范围，以便创建神经网络参数的忠实编码。与块浮点、均匀、类似IEEE的浮点或浮点编码相比，AdaptivFloat在各种最先进的神经网络拓扑结构中以极低的精度（$\leq$ 8位）始终产生更高的推理精度。值得注意的是，在$\leq$ 8位的权重位宽下，AdaptivFloat的BLEU得分高达+0.3，字出错率为-0.75，超过了基准FP 32性能。在深度神经网络（DNN）硬件加速器上的实验结果，在其计算数据路径中利用AdaptivFloat逻辑，证明了每操作能量和面积分别为0.9 $\times$和1.14 $\times$，即基于等效位宽整数的加速器变体的能量和面积。
英文摘要：
Conventional hardware-friendly quantization methods, such as fixed-point or integer, tend to perform poorly at very low word sizes as their shrinking dynamic ranges cannot adequately capture the wide data distributions commonly seen in sequence transduction models. We present AdaptivFloat, a floating-point inspired number representation format for deep learning that dynamically maximizes and optimally clips its available dynamic range, at a layer granularity, in order to create faithful encoding of neural network parameters. AdaptivFloat consistently produces higher inference accuracies compared to block floating-point, uniform, IEEE-like float or posit encodings at very low precision ($\leq$ 8-bit) across a diverse set of state-of-the-art neural network topologies. And notably, AdaptivFloat is seen surpassing baseline FP32 performance by up to +0.3 in BLEU score and -0.75 in word error rate at weight bit widths that are $\leq$ 8-bit. Experimental results on a deep neural network (DNN) hardware accelerator, exploiting AdaptivFloat logic in its computational datapath, demonstrate per-operation energy and area that is 0.9$\times$ and 1.14$\times$, respectively, that of equivalent bit width integer-based accelerator variants.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
