arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.06178cs.PFcs.AIcs.DCcs.LG
收藏

Look-Up mAI GeMM:通过 msGeMM 将 AI GeMM 性能提升近 2.5 倍

Look-Up mAI GeMM: Increasing AI GeMMs Performance by Nearly 2.5x via msGeMM

Microsoft Research(微软研究院)

Saeed Maleki

AI总结:

本文提出 msGeMM 算法,利用特殊 CUDA 核心从查找表添加元素,使低精度 AI 模型乘加指令减少约 2.5 倍,显著提升 GeMM 性能。

摘要:

AI 模型规模不断增大,社区近期进展表明,与需要双精度数据类型的高性能计算(HPC)应用不同,fp8 或 int4 等低精度数据类型足以在训练和推理中保持相同的模型质量。顺应这些趋势,NVIDIA 和 AMD 等 GPU 厂商通过张量核心(Tensor Cores)为 fp16、fp8 和 int8 GeMM 操作提供了硬件支持,并实现了卓越性能。然而,本文提出了一种名为 msGeMM 的新算法,表明采用低精度数据类型的 AI 模型可以减少约 2.5 倍的乘法和加法指令。该算法的高效实现需要特殊的 CUDA 核心,能够以张量核心的速率从小型查找表中添加元素。

英文摘要:

AI models are increasing in size and recent advancement in the community has shown that unlike HPC applications where double precision datatype are required, lower-precision datatypes such as fp8 or int4 are sufficient to bring the same model quality both for training and inference. Following these trends, GPU vendors such as NVIDIA and AMD have added hardware support for fp16, fp8 and int8 GeMM operations with an exceptional performance via Tensor Cores. However, this paper proposes a new algorithm called msGeMM which shows that AI models with low-precision datatypes can run with ~2.5x fewer multiplication and add instructions. Efficient implementation of this algorithm requires special CUDA cores with the ability to add elements from a small look-up table at the rate of Tensor Cores.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑