Kitsune:在GPU上实现数据流执行
Kitsune: Enabling Dataflow Execution on GPUs
AI总结:
Kitsune通过适度调整GPU架构并基于PyTorch Dynamo构建编译器,实现高效数据流执行,在5个应用中提升性能1.1-2.4倍并减少16%-98%片外流量。
摘要:
最先进的深度学习模型在规模和复杂度上不断增长,许多现代模型在行为上也日益异构化。GPU仍是深度学习应用的主导平台,但其依赖的批量同步执行模型存在诸多缺陷,且不适合深度学习应用的图结构。许多工业界和学术界的工作试图通过垂直融合来克服这些问题,但这种方法仍未能实现三个未开发的机会:(1)由于SM的时间多路复用,GPU上许多资源在仅执行单个算子时处于空闲状态;(2)更智能的片上数据移动可降低能耗,从而在功率受限环境中带来更高性能;(3)无法利用隐藏维度或归约维度作为并行性来源,以缓解批大小压力。本文探索了相对未开发的领域,回答以下关键问题:对当前GPU架构进行适度调整能否实现高效的数据流执行,从而规避垂直融合的限制,而无需重新设计全新架构。我们开发了Kitsune——一组在GPU上实现数据流执行的原语,以及一个基于PyTorch Dynamo的端到端编译器。在5个挑战性应用中,Kitsune在推理和训练方面分别提供了1.3倍至2.3倍和1.1倍至2.4倍的性能提升,以及41%至98%和16%至42%的片外流量减少。
英文摘要:
State of art DL models are growing in size and complexity, with many modern models also increasing in heterogeneity of behavior. GPUs are still the dominant platform for DL applications, relying on a bulk-synchronous execution model which has many drawbacks and is ill-suited for the graph structure of DL applications. Many industry and academic works attempt to overcome these by employing vertical fusion but this approach still fails to realize three untapped opportunities: (1) the fact that many resources on the GPU are idle while only one operator executes due to temporal multiplexing of the SM; (2) lower energy from more intelligent on-chip data-movement which lends to higher performance in a power-provisioned environment. (3) inability to exploit hidden or reduction dimensions as a source of parallelism to ease pressure on batch size. This paper explores relatively uncharted territory, answering the following key question: Can modest adjustments to the current GPU architecture enable efficient dataflow execution, thereby circumventing the constraints of vertical fusion without necessitating a clean-slate architecture design. We develop Kitsune -- a set of primitives that enable dataflow execution on GPUs and an end-to-end compiler based on PyTorch Dynamo. Across 5 challenge applications, Kitsune can provide 1.3$\times$-2.3$\times$ and 1.1$\times$-2.4$\times$ performance improvement as well as 41%-98% and 16%-42% off-chip traffic reduction for inference and training, respectively.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。