arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2206.02874cs.AR
收藏

通过微基准剖析张量核心:延迟、吞吐量和数值行为

Dissecting Tensor Cores via Microbenchmarks: Latency, Throughput and Numeric Behaviors

Wei Sun, Ang Li, Tong Geng, Sander Stuijk, Henk Corporaal

摘要:

自Volta架构以来,张量核心一直是所有NVIDIA GPU中加速融合矩阵乘法累加(MMA)的重要单元。要对Tensor Cores进行编程,用户必须使用旧版wmma API或当前的mma API。传统的wmma API更易于使用,但只能利用Tensor Cores的有限功能和功能。具体来说,wmma API支持更少的操作数形状,并且无法利用最新Ampere Tensor Cores的新稀疏矩阵乘法功能。然而,目前的编程接口的性能还没有得到很好的探讨。此外,最新的Ampere Tensor Core支持的低精度浮点(TF32,BF 16和FP 16)的计算数值行为也是神秘的。在本文中,我们将探讨当前编程API的吞吐量和延迟。我们还直观地研究了张量核MMA的数值行为,并描述了中间操作,包括乘法,内积加法和累加。本工作中使用的所有代码都可以在https://github.com/sunlex0717/DissectingTensorCores中找到。

英文摘要:

Tensor Cores have been an important unit to accelerate Fused Matrix Multiplication Accumulation (MMA) in all NVIDIA GPUs since Volta Architecture. To program Tensor Cores, users have to use either legacy wmma APIs or current mma APIs. Legacy wmma APIs are more easy-to-use but can only exploit limited features and power of Tensor Cores. Specifically, wmma APIs support fewer operand shapes and can not leverage the new sparse matrix multiplication feature of the newest Ampere Tensor Cores. However, the performance of current programming interface has not been well explored. Furthermore, the computation numeric behaviors of low-precision floating points (TF32, BF16, and FP16) supported by the newest Ampere Tensor Cores are also mysterious. In this paper, we explore the throughput and latency of current programming APIs. We also intuitively study the numeric behaviors of Tensor Cores MMA and profile the intermediate operations including multiplication, addition of inner product, and accumulation. All codes used in this work can be found in https://github.com/sunlex0717/DissectingTensorCores.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑