Collage:深度学习后端与自动放置的无缝集成
Collage: Seamless Integration of Deep Learning Backends with Automatic Placement
摘要:
对高效、高性能部署深度学习(DL)应用的强烈需求推动了丰富的DL生态系统的快速发展。为了跟上这种快速发展,现代DL框架必须有效地集成各种优化的张量代数库和运行时作为其后端,并使用这些后端生成最快的可执行文件。然而,当前的DL框架需要大量的人工工作和专业知识来集成每个新的后端,而无法释放其全部潜力。鉴于深度学习生态系统的快速发展特性,这种手动方法通常会减缓跨不同层的持续创新;它阻碍了硬件供应商快速部署其尖端库,DL框架开发人员必须反复调整其手工编码的规则以适应新版本的库,机器学习从业者需要等待新技术的集成,并且经常遇到不满意的性能。在本文中,我们提出拼贴,DL框架,提供DL后端的无缝集成。Collage提供了一个富有表现力的后端注册接口,允许用户精确地指定各种后端的功能。通过利用可用后端的规范,Collage自动搜索针对给定工作负载和执行环境的优化后端放置策略。我们的评估显示,Collage在NVIDIA的RTX 2070 GPU、V100 GPU和英特尔的Xeon 8259 CL CPU上的表现,平均分别超过了每种硬件的现有最佳框架$1.26\times$、$1.43\times$和$1.40\times$。Collage已经开源并部署在Apache TVM中。
英文摘要:
The strong demand for efficient and performant deployment of Deep Learning (DL) applications prompts the rapid development of a rich DL ecosystem. To keep up with this fast advancement, it is crucial for modern DL frameworks to efficiently integrate a variety of optimized tensor algebra libraries and runtimes as their backends and generate the fastest possible executable using these backends. However, current DL frameworks require significant manual effort and expertise to integrate every new backend while failing to unleash its full potential. Given the fast-evolving nature of the DL ecosystem, this manual approach often slows down continuous innovations across different layers; it prevents hardware vendors from the fast deployment of their cutting-edge libraries, DL framework developers must repeatedly adjust their hand-coded rules to accommodate new versions of libraries, and machine learning practitioners need to wait for the integration of new technologies and often encounter unsatisfactory performance. In this paper, we propose Collage, a DL framework that offers seamless integration of DL backends. Collage provides an expressive backend registration interface that allows users to precisely specify the capability of various backends. By leveraging the specifications of available backends, Collage automatically searches for an optimized backend placement strategy for a given workload and execution environment. Our evaluation shows that Collage outperforms the best existing framework for each hardware by $1.26\times$, $1.43\times$, $1.40\times$ on average on NVIDIA's RTX 2070 GPU, V100 GPU, and Intel's Xeon 8259CL CPU, respectively. Collage has been open-sourced and deployed in Apache TVM.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。