从Roofline到Ruggedness:分解和平滑GEMM性能景观
From Roofline to Ruggedness: Decomposing and Smoothing the GEMM Performance Landscape
AI总结:
针对GPU上GEMM性能的锯齿状波动(相邻问题吞吐量差异达30%),提出性能粗糙度分析框架,通过最佳动态分块选择和基于动态规划的填充分割优化器,将粗糙度降低70%并提升平均吞吐量30%。
摘要:
相邻的GEMM问题在N上仅差128步,在相同GPU上吞吐量可相差30%。这种普遍存在的性能粗糙度——对Roofline分析和峰值FLOPs直觉不可见,但对每个非峰值工作负载占主导——是本文的主题。 我们提出性能粗糙度分析作为Roofline的补充分析框架:不将GPU性能总结为标量界限,而是将完整的多维性能表面作为研究对象,将其纹理分解为可归因于机制的组件,并将软件可移除的贡献与硬件绑定的贡献分开。该框架直接类比深度学习损失景观——一个连续量(理想时间2MNK / 计算吞吐量峰值)因与离散硬件基板(分块、子组、缓存行、DRAM通道)的交互而变得粗糙。 我们通过32768个配置的扫描(M, N, K)属于{128, ..., 4096}^3,将框架应用于Intel Battlemage(Arc B580, sycl-tla)上的BF16 NN(无转置)GEMM。峰值在非正方形形状M=3840, N=2048, K=4096且默认分块大小时为110.8 TFLOPs;初始景观粗糙度为每128步16.8 TFLOPs,而理想值为2.0。一个两阶段软件栈——(i) 最佳六种动态分块选择和(ii) 一种新颖的基于动态规划的填充分割优化器,具有O(1)运行时查找——将粗糙度降低70%并将平均吞吐量提升30%。跨分块实验表明,残余锯齿周期与软件分块大小完全成比例,排除了缓存集冲突,并将剩余方差归因于四个硬件绑定来源(每内核基础开销、波前量化、DPAS原子几何和GDDR6通道哈希交互)。
英文摘要:
Adjacent GEMM problems that differ by a single 128-element step in N can show 30% different throughput. This pervasive performance ruggedness - invisible to roofline analysis and peak-FLOPs intuition, yet dominant for every non-peak workload - is the subject of this paper. We propose performance ruggedness analysis, an analytical framework complementary to roofline: rather than summarizing a GPU with a scalar bound, it treats the full multidimensional performance surface as the object of study, decomposes its texture into mechanism-attributable components, and separates software-removable from hardware-bound losses. The framing is analogous to deep-learning loss landscapes: a continuous quantity (idealized time 2MNK/peak) made rugged by discrete hardware substrates (tiles, sub-groups, cache lines, DRAM channels). We instantiate it on BF16 NN GEMM on Intel Battlemage (Arc B580, sycl-tla) via a 32,768-configuration sweep over (M,N,K) in {128,...,4096}^3. We introduce roughness, the mean absolute step-to-step throughput change, which starts at 16.8 TFLOPs/128-step against an ideal of 2.0. A two-stage stack - best-of-six dynamic tile selection and a novel dynamic-programming padding-and-splitting optimizer (precomputed once, O(1) at runtime) - cuts roughness by 70% and raises mean throughput by 30%. Cross-tile experiments show the residual sawtooth period scales exactly with the tile size, ruling out cache conflicts and attributing the rest to four hardware-bound sources. Finally, we derive the optimal achievable landscape from first principles - datasheet integers alone, no kernel run or simulator - and turn it into an optimality scale (Kernel Optimality Levels) grading any kernel by how much of that landscape it attains and how close its roughness lies to the hardware floor; the production kernel and our optimized stack rate L0 and L2 despite both reporting ~95% of peak.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。