CoCoPIE XGen:一个面向AI的全栈优化框架
CoCoPIE XGen: A Full-Stack AI-Oriented Optimizing Framework
摘要:
越来越多的人需要将人工智能功能从云上的数据中心转移到边缘或终端设备,例如在智能手机,AR/VR设备,自动驾驶汽车和各种物联网设备上运行的快速新兴的实时AI应用程序。然而,DNN计算需求与边缘或终端设备上的计算能力之间的巨大差距严重阻碍了这一转变。本文介绍了XGen的设计,这是一个DNN的优化框架,旨在弥合这一差距。XGen将跨领域协同设计作为首要考虑因素。其面向AI的全栈优化包括DNN软件栈每一层的许多创新优化,所有这些优化都是以合作的方式设计的。独特的技术使XGen能够优化各种DNN,包括那些具有极端深度的DNN(例如,BERT、GPT和其他Transformers),并生成比现有DNN框架运行速度快几倍的代码,同时提供相同的准确度。
英文摘要:
There is a growing demand for shifting the delivery of AI capability from data centers on the cloud to edge or end devices, exemplified by the fast emerging real-time AI-based apps running on smartphones, AR/VR devices, autonomous vehicles, and various IoT devices. The shift has however been seriously hampered by the large growing gap between DNN computing demands and the computing power on edge or end devices. This article presents the design of XGen, an optimizing framework for DNN designed to bridge the gap. XGen takes cross-cutting co-design as its first-order consideration. Its full-stack AI-oriented optimizations consist of a number of innovative optimizations at every layer of the DNN software stack, all designed in a cooperative manner. The unique technology makes XGen able to optimize various DNNs, including those with an extreme depth (e.g., BERT, GPT, other transformers), and generate code that runs several times faster than those from existing DNN frameworks, while delivering the same level of accuracy.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。