arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.20204cs.PLcs.AI
收藏

分析基于MLIR的AI内核编译器中的延迟隐藏与并行性

Analyzing Latency Hiding and Parallelism in an MLIR-based AI Kernel Compiler

Qualcomm Technologies International, Ltd.(高通技术国际有限公司);Qualcomm Technologies, Inc.(高通技术公司)

Javed Absar, Samarth Narang, Muthu Baskaran

AI总结:

本文研究了基于MLIR的AI内核编译器中延迟隐藏与并行性的优化方法,通过实验验证了向量化、多线程和双缓冲技术对不同内核性能的影响。

摘要:

边缘设备上的AI内核编译依赖于编译器在存在分层内存和显式数据移动的情况下利用并行性和隐藏内存延迟的能力。本文报告了一种基准测试方法及其结果,针对基于MLIR的编译流水线中的三种编译器控制机制:向量化(Vec)、跨硬件上下文的多线程(MT)以及使用ping-pong暂存缓冲区的双缓冲(DB)以重叠DMA传输与计算。使用Triton/Inductor生成的内核,我们提出了一个消融阶梯,以分离Vec、MT和DB的贡献,并量化MT加速如何随着问题规模而变化,使用GELU作为代表激活内核。结果表明,向量化为带宽敏感内核提供了主要增益,MT在调度开销摊销后提供显著改进,而DB在传输和计算可以重叠(即在纯内存受限或纯计算受限行为之外)时提供额外益处。

英文摘要:

AI kernel compilation for edge devices depends on the compiler's ability to exploit parallelism and hide memory latency in the presence of hierarchical memory and explicit data movement. This paper reports a benchmark methodology and corresponding results for three compiler-controlled mechanisms in an MLIR-based compilation pipeline: vectorization (Vec), multi-threading (MT) across hardware contexts, and double buffering (DB) using ping--pong scratchpad buffers to overlap DMA transfers with compute. Using Triton/Inductor-generated kernels, we present an ablation ladder that separates the contribution of Vec, MT, and DB, and we quantify how MT speedup scales with problem size using GELU as a representative activation kernel. The results show that vectorization provides the primary gain for bandwidth-sensitive kernels, MT delivers substantial improvements once scheduling overhead is amortized, and DB provides additional benefit when transfers and compute can be overlapped (i.e., outside the extremes of purely memory-bound or purely compute-bound behavior).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑