高精度可扩展微缩数据路径与优化缩减树用于高效NPU集成
Precision-Scalable Microscaling Datapaths with Optimized Reduction Tree for Efficient NPU Integration
AI总结：
本文提出了一种混合精度可扩展的缩减树用于MX MACs，以实现高效混合精度累积，并将其集成到SNAX平台中，提高了NPU的能效和吞吐量。
摘要：
新兴的持续学习应用需要下一代神经处理单元（NPU）平台来支持训练和推理操作。有前途的微缩（MX）标准使推理能够使用窄位宽，而训练则能提供大的动态范围。然而，现有的MX乘加（MAC）设计面临一个关键的权衡：整数累积需要昂贵的转换从窄浮点数乘积，而FP32累积则遭受量化损失和昂贵的归一化。为了解决这些限制，我们提出了一种混合精度可扩展的缩减树用于MX MACs，结合了两种方法的优点，实现了高效的混合精度累积，具有受控的精度放松。此外，我们将这些MACs的8x8阵列集成到最先进的（SotA）NPU集成平台SNAX中，以提供高效的控制和数据传输到我们优化的精度可扩展MX数据路径。我们对设计进行了MAC和系统级的评估，并将其与SotA进行比较。我们的集成系统在MXINT8、MXFP8/6和MXFP4上分别实现了657、1438-1675和4065 GOPS/W的能量效率，吞吐量分别为64、256和512 GOPS。
英文摘要：
Emerging continual learning applications necessitate next-generation neural processing unit (NPU) platforms to support both training and inference operations. The promising Microscaling (MX) standard enables narrow bit-widths for inference and large dynamic ranges for training. However, existing MX multiply-accumulate (MAC) designs face a critical trade-off: integer accumulation requires expensive conversions from narrow floating-point products, while FP32 accumulation suffers from quantization losses and costly normalization. To address these limitations, we propose a hybrid precision-scalable reduction tree for MX MACs that combines the benefits of both approaches, enabling efficient mixed-precision accumulation with controlled accuracy relaxation. Moreover, we integrate an 8x8 array of these MACs into the state-of-the-art (SotA) NPU integration platform, SNAX, to provide efficient control and data transfer to our optimized precision-scalable MX datapath. We evaluate our design both on MAC and system level and compare it to the SotA. Our integrated system achieves an energy efficiency of 657, 1438-1675, and 4065 GOPS/W, respectively, for MXINT8, MXFP8/6, and MXFP4, with a throughput of 64, 256, and 512 GOPS.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
