arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2209.02341cs.LGcs.DC
收藏

EnergonAI:一个100 - 1000亿参数Transformer模型的推理系统

EnergonAI: An Inference System for 10-100 Billion Parameter Transformer Models

HPC-AI Technology Inc.(HPC-AI科技公司);Sun Yat-sen University(中山大学);National University of Singapore(新加坡国立大学)

Jiangsu Du, Ziming Liu, Jiarui Fang, Shenggui Li, Yongbin Li, Yutong Lu, Yang You

摘要:

大型Transformer模型在广泛的自然语言处理(NLP)任务上显示出良好的性能。尽管AI社区已经将模型规模扩展到万亿参数级别,但由于延迟,吞吐量和内存限制,100 - 1000亿参数模型的实际部署仍然不确定。在本文中,我们提出了EnergonAI,以解决在单GPU或多GPU系统上高效部署100 - 1000亿参数Transformer模型的挑战。EnergonAI采用分层控制器系统架构来协调多个设备,并有效支持不同的并行模式。它以单控制器的方式将子模型的执行委托给多个工作者,并以多控制器的方式在工作者之间应用张量并行和流水线并行。在此基础上,提出了三种新的技术,即非阻塞流水线并行、分布式冗余计算消除和对等内存池。EnergonAI使用户能够像编写串行代码一样编写复杂的并行代码。与FasterTransformer相比,我们已经证明EnergonAI在延迟和吞吐量方面具有卓越的性能。在我们的实验中,EnergonAI可以在张量并行中实现37%的延迟减少,在流水线并行中实现10%的可扩展性改进,并且通过使用更大的异构内存空间来提高在单个GPU上推断的模型规模,但性能降低有限。

英文摘要:

Large transformer models display promising performance on a wide range of natural language processing (NLP) tasks. Although the AI community has expanded the model scale to the trillion parameter level, the practical deployment of 10-100 billion parameter models is still uncertain due to the latency, throughput, and memory constraints. In this paper, we proposed EnergonAI to solve the challenges of the efficient deployment of 10-100 billion parameter transformer models on single- or multi-GPU systems. EnergonAI adopts a hierarchy-controller system architecture to coordinate multiple devices and efficiently support different parallel patterns. It delegates the execution of sub-models to multiple workers in the single-controller style and applies tensor parallelism and pipeline parallelism among the workers in a multi-controller style. Upon the novel architecture, we propose three techniques, i.e. non-blocking pipeline parallelism, distributed redundant computation elimination, and peer memory pooling. EnergonAI enables the users to program complex parallel code the same as a serial one. Compared with the FasterTransformer, we have proven that EnergonAI has superior performance on latency and throughput. In our experiments, EnergonAI can achieve 37% latency reduction in tensor parallelism, 10% scalability improvement in pipeline parallelism, and it improves the model scale inferred on a single GPU by using a larger heterogeneous memory space at cost of limited performance reduction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑