arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.13938cs.LGcs.AIcs.ARcs.NAmath.NA
收藏

eXmY:一种用于任意位精度量化的数据类型与技术

eXmY: A Data Type and Technique for Arbitrary Bit Precision Quantization

Google LLC(谷歌公司)

Aditya Agrawal, Matthew Hedlund, Blake Hechtman

AI总结:

eXmY 提出一种支持任意位宽与整数/浮点格式的新型量化数据类型及编解码技术,实现完美压缩、字节可寻址,并已在生产中部署近两年。

摘要:

eXmY 是一种用于机器学习模型量化的新型数据类型。它支持任意位宽以及任意的整数和浮点格式。例如,它无缝支持 3、5、6、7、9 位格式。对于特定的位宽,比如 7 位,它定义了所有可能的格式,例如 e0m6、e1m5、e2m4、e3m3、e4m2、e5m1 和 e6m0。对于非 2 的幂次方位宽(如 5、6、7 位),我们创建了一种新颖的编码和解码方案,该方案实现了完美压缩、字节可寻址性,并且适用于分片和向量处理。我们在 C++、TensorFlow、JAX 和 PAX 中实现了用于模拟、编码和解码张量及检查点的库。为了获得最佳性能,编解码器在 CPU 上使用 SIMD 指令,在 TPU 和 GPU 上使用向量指令。eXmY 还是一种技术,它利用了张量中指数的统计分布。它可用于量化权重、静态和动态激活、梯度、主权重和优化器状态。它可以减少内存(CPU DRAM 和加速器 HBM)、网络和磁盘存储及传输。它可以提高多租户能力并加速计算。eXmY 已在生产中部署近两年。

英文摘要:

eXmY is a novel data type for quantization of ML models. It supports both arbitrary bit widths and arbitrary integer and floating point formats. For example, it seamlessly supports 3, 5, 6, 7, 9 bit formats. For a specific bit width, say 7, it defines all possible formats e.g. e0m6, e1m5, e2m4, e3m3, e4m2, e5m1 and e6m0. For non-power of two bit widths e.g. 5, 6, 7, we created a novel encoding and decoding scheme which achieves perfect compression, byte addressability and is amenable to sharding and vector processing. We implemented libraries for emulation, encoding and decoding tensors and checkpoints in C++, TensorFlow, JAX and PAX. For optimal performance, the codecs use SIMD instructions on CPUs and vector instructions on TPUs and GPUs. eXmY is also a technique and exploits the statistical distribution of exponents in tensors. It can be used to quantize weights, static and dynamic activations, gradients, master weights and optimizer state. It can reduce memory (CPU DRAM and accelerator HBM), network and disk storage and transfers. It can increase multi tenancy and accelerate compute. eXmY has been deployed in production for almost 2 years.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑