使用Ozaki-II方案进行矩阵乘法仿真的误差分析
Error Analysis of Matrix Multiplication Emulation Using Ozaki-II Scheme
AI总结:
本文对Ozaki-II方案进行了严格的确定性误差分析,揭示了其精度行为并提供了估算所需低精度矩阵乘法次数的方法。
摘要:
Ozaki-II方案是一种利用中国剩余定理通过一系列低精度矩阵乘法来计算高精度矩阵乘法的仿真方法。在该方案中,可达到的数值精度随着低精度矩阵乘法次数的增加而提高。先前的数值研究表明,使用Ozaki-II方案进行单精度和双精度矩阵乘法在现代AI硬件上比标准BLAS例程具有更高的吞吐量,这些硬件配备了具有INT8输入和INT32累积的快速INT8矩阵乘法累积单元。然而,当输入矩阵的指数分布较宽时,Ozaki-II方案的精度会下降,此时需要大量的低精度矩阵乘法运算才能获得高精度结果。在本文中,我们对Ozaki-II方案进行了严格的确定性误差分析。所提出的分析不仅阐明了该方法的精度行为,还能够估计出实现所需数值精度所需的低精度矩阵乘法次数。
英文摘要:
The Ozaki-II scheme is an emulation method that leverages the Chinese Remainder Theorem to compute high-precision matrix multiplication via a sequence of low-precision matrix multiplications. In this scheme, the attainable numerical accuracy improves as the number of low-precision matrix multiplications increases. Previous numerical studies have shown that single- and double-precision matrix multiplication using the Ozaki-II scheme achieves higher throughput than that of standard BLAS routines on modern AI hardware equipped with fast INT8 matrix multiply-accumulate units with INT8 inputs and INT32 accumulation. However, the accuracy of the Ozaki-II scheme can degrade when the exponent distribution of the input matrices is wide, in which case a large number of low-precision matrix multiplications is required to obtain high-precision results. In this paper, we present a rigorous deterministic error analysis of the Ozaki-II scheme. The proposed analysis not only clarifies the accuracy behavior of the method but also enables the estimation of the number of low-precision matrix multiplications required to achieve a desired level of numerical accuracy.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。