arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.13853cs.LGcs.PF
收藏

面向深度学习训练与推理的GPU性能预测

Forecasting GPU Performance for Deep Learning Training and Inference

Georgia Institute of Technology(佐治亚理工学院);Microsoft Research(微软研究院);Meta

Seonho Lee, Amar Phanishayee, Divya Mahajan

AI总结:

针对难以获取新GPU、无法快速评估模型与GPU适配性能的问题,提出NeuSight框架,通过瓦片分解结合机器学习与性能定律预测未见过的GPU上的深度学习训练推理延迟,大幅降低预测误差。

摘要:

深度学习核函数具有可预测的内存访问与计算模式,这使得GPU的并行架构非常适合执行这类任务。GPU的软件与运行时系统经过优化,可更好地利用流多处理器、片上缓存以及片外高带宽内存。随着深度学习模型与GPU的不断演进,人们往往难以获取最新款GPU,由此引发了三类问题:新模型架构在现有GPU上的性能如何、现有模型在新GPU上的性能如何,以及新模型架构在新GPU上的性能如何。为解答这些问题,我们提出了NeuSight(一种深度学习性能预测框架),它无需实际执行,即可预测各类深度学习模型在未见过的GPU上的训练与推理性能。该框架结合GPU硬件行为与软件库优化来估算端到端性能。此前的研究采用捕捉线性趋势的回归模型或多层感知机,来预测深度学习核函数在GPU上的整体延迟。当预测未见过的模型与新GPU上的性能时,这些方法的误差百分比更高。与之不同,NeuSight将预测问题分解为多个更小的子问题,并通过基础性能定律对预测结果加以约束。NeuSight将单个深度学习核函数的预测任务拆解为被称为tile(瓦片)的更小工作集,这些工作集在GPU上独立执行。它采用机器学习方法确定瓦片粒度的预测结果,再将这些结果聚合以估算端到端延迟。在各类深度学习工作负载与最新GPU上,NeuSight的表现均优于此前的研究。在预测GPT3模型在H100上的训练与推理延迟时,与当前最优的已有工作相比,NeuSight将误差百分比从121.4%和30.8%降至2.3%,且训练该框架时并未使用GPT3和H100相关数据。

英文摘要:

Deep learning kernels exhibit predictable memory accesses and compute patterns, making GPUs' parallel architecture well-suited for their execution. Software and runtime systems for GPUs are optimized to better utilize the stream multiprocessors, on-chip cache, and off-chip high-bandwidth memory. As deep learning models and GPUs evolve, access to newer GPUs is often limited, raising questions about the performance of new model architectures on existing GPUs, existing models on new GPUs, and new model architectures on new GPUs. To address these questions, we introduce NeuSight, a framework to predict the performance of various deep learning models, for both training and inference, on unseen GPUs without requiring actual execution. The framework leverages both GPU hardware behavior and software library optimizations to estimate end-to-end performance. Previous work uses regression models that capture linear trends or multilayer perceptrons to predict the overall latency of deep learning kernels on GPUs. These approaches suffer from higher error percentages when forecasting performance on unseen models and new GPUs. Instead, NeuSight decomposes the prediction problem into smaller problems, bounding the prediction through fundamental performance laws. NeuSight decomposes a single deep learning kernel prediction into smaller working sets called tiles, which are executed independently on the GPU. Tile-granularity predictions are determined using a machine learning approach and aggregated to estimate end-to-end latency. NeuSight outperforms prior work across various deep learning workloads and the latest GPUs. It reduces the percentage error from 121.4% and 30.8% to 2.3% in predicting the latency of GPT3 model for training and inference on H100, compared to state-of-the-art prior work, where both GPT3 and H100 were not used to train the framework.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑