一种在低比特 GEMM 残差计算中使用 RSVD 的方法
A method of using RSVD in residual calculation of LowBit GEMM
AI总结：
提出LRQMM方法，通过在低精度量化矩阵乘法残差补偿中引入低秩近似，在仅增加BLAS-2级开销下将误差降低1~2个数量级，显著提升了深度学习网络中的计算精度。
摘要：
近年来硬件技术的进步为低精度应用带来了许多可能性。然而，使用低精度会引入显著的计算误差，对维持计算精度构成了相当大的挑战。我们提出了低秩残差量化矩阵乘法（LRQMM）方法，该方法在密集低精度量化矩阵乘法的残差补偿中引入了低秩近似。它只需 BLAS-2 级别的额外时间开销，即可带来数倍的精度提升。此外，LRQMM 是一种完全无数据的量化方法，不需要额外数据进行预训练。它仅作用于低精度 GEMM 算子，易于与其他方法结合。通过实验，LRQMM 能将直接量化矩阵乘法的误差降低 1~2 个数量级，在处理较大矩阵尺寸时，计算速度仅降低约 20%。在深度学习网络中，LRQMM-4bit 在 Resnet-50 上实现了 61.8% 的 ImageNet Top-1 准确率，而直接量化的准确率仅为 8.3%。
英文摘要：
The advancements of hardware technology in recent years has brought many possibilities for low-precision applications. However, the use of low precision can introduce significant computational errors, posing a considerable challenge to maintaining the computational accuracy. We propose low-rank residuals quantized matrix multiplication(LRQMM) method which introduces low-rank approximation in residual compensation for dense low precision quantization matrix multiplication. It can bring several times accuracy improvement with only BLAS-2 level extra time overhead. Moreover, LRQMM is a completely data-free quantization method that does not require additional data for pre-training. And it only works with low precision GEMM operator, which is easy to couple with other methods. Through experimentation, LRQMM can reduce the error of direct quantized matrix multiplication by 1~2 orders of magnitude, when dealing with larger matrix sizes, the computational speed is only reduced by approximately 20\%. In deep learning networks, LRQMM-4bit achieves 61.8% ImageNet Top-1 accuracy in Resnet-50, while the Direct Quant accuracy is only 8.3%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
