Hexagon-MLIR:一种面向高通神经处理单元(NPUs)的AI编译栈
Hexagon-MLIR: An AI Compilation Stack For Qualcomm's Neural Processing Units (NPUs)
AI总结:
Hexagon-MLIR是一种基于MLIR框架的开源编译栈,用于加速NPU上的AI工作负载,通过生成mega内核优化数据局部性,减少带宽瓶颈。
摘要:
本文介绍了Hexagon-MLIR,一个面向高通Hexagon神经处理单元(NPU)的开源编译栈,提供了统一的支持,用于将Triton内核和PyTorch模型进行下放。该编译器基于MLIR框架,通过一系列结构化的传递过程,利用NPU架构特性来加速AI工作负载。它使新的Triton内核(手写或来自PyTorch 2.0的子图)能够更快地部署到目标上,通过提供从内核到二进制的自动化编译。通过摄入Triton内核,我们生成了mega内核,以最大化NPU的紧密耦合内存(TCM)中的数据局部性,从而减少基于库方法固有的带宽瓶颈。这一举措通过提供一个基于MLIR的开源编译栈补充了我们的商业工具链,使开发人员能够通过更灵活的方法提升AI编译能力。Hexagon-MLIR仍处于进行中,我们继续在这一努力中添加许多更多的优化和功能。
英文摘要:
In this paper, we present Hexagon-MLIR,an open-source compilation stack that targets Qualcomm Hexagon Neural Processing Unit (NPU) and provides unified support for lowering Triton kernels and PyTorch models . Built using the MLIR framework, our compiler applies a structured sequence of passes to exploit NPU architectural features to accelerate AI workloads. It enables faster deployment of new Triton kernels (hand-written or subgraphs from PyTorch 2.0), for our target by providing automated compilation from kernel to binary. By ingesting Triton kernels, we generate mega-kernels that maximize data locality in the NPU's Tightly Coupled Memory (TCM), reducing the bandwidth bottlenecks inherent in library-based approaches. This initiative complements our commercial toolchains by providing developers with an open-source MLIR-based compilation stack that gives them a path to advance AI compilation capabilities through a more flexible approach. Hexagon-MLIR is a work-in-progress, and we are continuing to add many more optimizations and capabilities in this effort.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。