通过Microbenchmarking剖析NVidia Turing T4 GPU
Dissecting the NVidia Turing T4 GPU via Microbenchmarking
摘要:
在2019年,GPU制造商更新设计的速度很快,加上他们不愿透露微架构细节,对于那些希望提取尽可能高性能的软件设计师来说,仍然是一个障碍。去年,正是这些原因促使我们使用微基准测试来剖析Volta GPU架构。2018年8月,NVIDIA最新架构Turing的推出迫使我们更新我们的研究。在这份报告中,我们研究了图灵,并将其与前几代NVidia GPU进行了定量比较。具体来说,我们研究了T4 GPU:一个针对推理应用的低功耗板。我们描述了它的改进,其面向推理的前身:基于Pascal架构的P4 GPU。T4和P4 GPU的每瓦频率都比全尺寸GPU高得多。我们研究了T4的TensorCores的性能,发现低精度操作数的吞吐量比P4 GPU高得多。我们发现,图灵引入了新的指令,表达矩阵数学更简洁。我们映射图灵的指令空间,找到与Volta相同的编码和额外的指令。我们发现,图灵TU 104芯片具有相同的存储层次结构的深度作为伏GV 100的TU 104上的高速缓存级别的大小往往是两倍大的Pascal GP 104上发现的。我们对T4存储层次结构的每个组成部分进行了基准测试,发现其P4前身的整体性能有了很大的提高。我们研究了时钟调节如何影响达到功率或热量限制的计算密集型工作负载。我们的许多发现都是新颖的,在这里首次发表。所有这些都可以指导高性能软件开发人员更接近GPU的峰值性能。
英文摘要:
In 2019, the rapid rate at which GPU manufacturers refresh their designs, coupled with their reluctance to disclose microarchitectural details, is still a hurdle for those software designers who want to extract the highest possible performance. Last year, these very reasons motivated us to dissect the Volta GPU architecture using microbenchmarks. The introduction in August 2018 of Turing, NVidia's latest architecture, pressed us to update our study. In this report, we examine Turing and compare it quantitatively against previous NVidia GPU generations. Specifically, we study the T4 GPU: a low-power board aiming at inference applications. We describe its improvements against its inference-oriented predecessor: the P4 GPU based on the Pascal architecture. Both T4 and P4 GPUs achieve significantly higher frequency-per-Watt figures than their full-size counterparts. We study the performance of the T4's TensorCores, finding a much higher throughput on low-precision operands than on the P4 GPU. We reveal that Turing introduces new instructions that express matrix math more succinctly. We map Turing's instruction space, finding the same encoding as Volta, and additional instructions. We reveal that the Turing TU104 chip has the same memory hierarchy depth as the Volta GV100; cache levels sizes on the TU104 are frequently twice as large as those found on the Pascal GP104. We benchmark each constituent of the T4 memory hierarchy and find substantial overall performance improvements over its P4 predecessor. We studied how clock throttling affects compute-intensive workloads that hit power or thermal limits. Many of our findings are novel, published here for the first time. All of them can guide high-performance software developers get closer to the GPU's peak performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。