使用 Korch 实现张量程序的最优 Kernel 编排
Optimal Kernel Orchestration for Tensor Programs with Korch
AI总结:
本文提出张量程序优化器 Korch,通过算子分裂、原语级约束建模和二元线性规划求解最优 GPU kernel 编排,在 V100/A100 上最高提升 1.7/1.6 倍。
摘要:
Kernel 编排是指将深度神经网络(DNN)不同算子中定义的计算映射到现代硬件平台上 GPU kernel 执行的任务。以往方法通过贪婪地应用算子融合来优化 kernel 编排,即将多个算子的计算融合到单个 kernel 中,因而错失了 kernel 编排中的多种优化机会。本文提出 Korch,这是一个张量程序优化器,能够为张量程序发现最优的 kernel 编排策略。Korch 并不直接融合算子,而是首先应用算子分裂,将张量算子分解为一小组基本张量代数原语。这种分解支持多种细粒度的算子间优化。随后,Korch 将 kernel 编排形式化为一个约束优化问题,利用现成的二元线性规划求解器发现最优编排策略,并生成可直接部署在现代 GPU 平台上的可执行程序,从而优化 kernel 编排。在多种 DNN 上的评估表明,Korch 优于现有张量程序优化器:在 V100 GPU 上最高达到 1.7 倍,在 A100 GPU 上最高达到 1.6 倍。Korch 已公开发布于 https://github.com/humuyan/Korch。
英文摘要:
Kernel orchestration is the task of mapping the computation defined in different operators of a deep neural network (DNN) to the execution of GPU kernels on modern hardware platforms. Prior approaches optimize kernel orchestration by greedily applying operator fusion, which fuses the computation of multiple operators into a single kernel, and miss a variety of optimization opportunities in kernel orchestration. This paper presents Korch, a tensor program optimizer that discovers optimal kernel orchestration strategies for tensor programs. Instead of directly fusing operators, Korch first applies operator fission to decompose tensor operators into a small set of basic tensor algebra primitives. This decomposition enables a diversity of fine-grained, inter-operator optimizations. Next, Korch optimizes kernel orchestration by formalizing it as a constrained optimization problem, leveraging an off-the-shelf binary linear programming solver to discover an optimal orchestration strategy, and generating an executable that can be directly deployed on modern GPU platforms. Evaluation on a variety of DNNs shows that Korch outperforms existing tensor program optimizers by up to 1.7x on V100 GPUs and up to 1.6x on A100 GPUs. Korch is publicly available at https://github.com/humuyan/Korch.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。