arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.12415cs.LGcs.DC
收藏

ALT:打破图级和运算符级优化之间的壁垒,提升深度学习性能

ALT: Boosting Deep Learning Performance by Breaking the Wall between Graph and Operator Level Optimizations

Nanjing University(南京大学);Huawei Technologies(华为技术有限公司);The Hong Kong Polytechnic University(香港理工大学)

Zhiying Xu, Jiafan Xu, Hongding Peng, Wei Wang, Xiaoliang Wang, Haoran Wan, Haipeng Dai, Yixu Xu, Hao Cheng, Kun Wang, Guihai Chen

摘要:

深度学习模型依赖于高度优化的张量库,以便在异构硬件上进行高效推理。当前的深度编译器通常预先确定张量的布局,然后优化运算符的循环。然而,这种单向和一次性的工作流严格地将图级优化和操作员级优化分离到不同的系统层中,从而错过了统一调优的机会。本文提出了ALT,一个编译器,执行联合图和操作符级的深度模型优化。ALT提供了一个通用的转换模块,通过简单易用的基本函数来操作布局和循环。ALT还集成了一个自动调优模块,可以在保证效率的同时,联合优化图形级数据布局和操作员级循环。实验结果表明,ALT的性能明显优于最先进的编译器(例如,Ansor)在单个操作者性能(例如,平均加速1.5倍)和端到端推理性能(例如,平均加速1.4倍)。

英文摘要:

Deep learning models rely on highly optimized tensor libraries for efficient inference on heterogeneous hardware. Current deep compilers typically predetermine layouts of tensors and then optimize loops of operators. However, such unidirectional and one-off workflow strictly separates graph-level optimization and operator-level optimization into different system layers, missing opportunities for unified tuning. This paper proposes ALT, a compiler that performs joint graph- and operator-level optimizations for deep models. ALT provides a generic transformation module to manipulate layouts and loops with easy-to-use primitive functions. ALT further integrates an auto-tuning module that jointly optimizes graph-level data layouts and operator-level loops while guaranteeing efficiency. Experimental results show that ALT significantly outperforms state-of-the-art compilers (e.g., Ansor) in terms of both single operator performance (e.g., 1.5x speedup on average) and end-to-end inference performance (e.g., 1.4x speedup on average).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑