arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1803.04014cs.DCcs.PF
收藏

NVIDIA张量核心可编程性、性能和精度

NVIDIA Tensor Core Programmability, Performance & Precision

Stefano Markidis, Steven Wei Der Chien, Erwin Laure, Ivy Bo Peng, Jeffrey S. Vetter

摘要:

NVIDIA Volta GPU微架构引入了一个名为“Tensor Core”的专用单元,每个时钟周期在4x 4矩阵上执行一次矩阵乘法和累加。NVIDIA Tesla V100加速器采用Volta微架构,提供640个Tensor核心,理论峰值性能为125 Tflops/s,混合精度。在本文中,我们调查目前的方法来编程NVIDIA张量核心,其性能和精度损失,由于计算混合精度。目前,NVIDIA提供了三种不同的在Tensor Cores上编程矩阵乘和累加的方法:CUDA Warp Matrix Multiply Accumulate(WMMA)API、CUTLASS(基于WMMA的模板库)和cuBLAS GEMM。在试验了不同的方法后,我们发现NVIDIA Tensor Cores在Tesla V100 GPU上的混合精度下可以提供高达83 Tflops/s的性能,分别是单精度和半精度性能的7倍和3倍。批量GEMM的WMMA实现达到了4 Tflops/s的性能。虽然由于半精度输入的矩阵乘法而导致的精度损失在许多HPC应用中可能是关键的,但可以以增加计算为代价来显著降低精度损失。我们的研究结果表明,使用矩阵乘法的HPC应用程序可以从NVIDIA Tensor Core中受益匪浅。

英文摘要:

The NVIDIA Volta GPU microarchitecture introduces a specialized unit, called "Tensor Core" that performs one matrix-multiply-and-accumulate on 4x4 matrices per clock cycle. The NVIDIA Tesla V100 accelerator, featuring the Volta microarchitecture, provides 640 Tensor Cores with a theoretical peak performance of 125 Tflops/s in mixed precision. In this paper, we investigate current approaches to program NVIDIA Tensor Cores, their performances and the precision loss due to computation in mixed precision. Currently, NVIDIA provides three different ways of programming matrix-multiply-and-accumulate on Tensor Cores: the CUDA Warp Matrix Multiply Accumulate (WMMA) API, CUTLASS, a templated library based on WMMA, and cuBLAS GEMM. After experimenting with different approaches, we found that NVIDIA Tensor Cores can deliver up to 83 Tflops/s in mixed precision on a Tesla V100 GPU, seven and three times the performance in single and half precision respectively. A WMMA implementation of batched GEMM reaches a performance of 4 Tflops/s. While precision loss due to matrix multiplication with half precision input might be critical in many HPC applications, it can be considerably reduced at the cost of increased computation. Our results indicate that HPC applications using matrix multiplications can strongly benefit from using of NVIDIA Tensor Cores.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑