arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.13474cs.DC
收藏

CompileMask:弥合注意力掩码的表达性与效率之间的差距

CompileMask: Bridging the Gap Between Expressiveness and Efficiency in Attention Masking

Beijing University of Posts and Telecommunications(北京邮电大学);Peking University(北京大学)

Hongzheng Li, Jiacheng Wan, Zhaolong Xing, Zhida Jiang, Yuanhang Yang, Junhua Shu, Jiaxing Wang, Xupeng Miao, Yingxia Shao, Zhen Chen, Ke Zhang

AI总结:

CompileMask 是一种弥合注意力掩码表达性与效率权衡的方法,通过编译器和 MaskSpec DSL 生成掩码预处理内核,实现比现有方法最高 3.8 倍的加速。

摘要:

现代注意力工作负载需要多样化的掩码模式,其边界取决于运行时元数据,但高效的注意力内核仅原生支持少量模式。现有在这些内核中支持自定义掩码的方法面临表达性与效率的权衡:基于谓词的方法表达性强,但会产生大量预处理和执行开销;基于区间的方法执行高效,但通过预定义的区间数量限制了表达性。我们提出 CompileMask,通过将表达性掩码规范与高效的注意力执行相结合来解决这一权衡问题。我们设计了一个编译器,在离线阶段分析可见性规则并生成特定于掩码的预处理内核;还提出了 MaskSpec,一种基于区域的领域特定语言(DSL),允许用户表达这些规则。在线阶段,生成的内核读取批次元数据并生成紧凑的执行描述符,使注意力内核能够通过将元素索引与预计算的端点进行比较来高效检查可见性。实验表明,CompileMask 比最先进的方法实现了高达 3.8 倍的加速。

英文摘要:

Modern attention workloads require diverse masking patterns whose boundaries depend on runtime metadata, yet efficient attention kernels natively support only a small set of patterns. Existing methods for supporting custom masks in these kernels face an expressiveness-efficiency trade-off. Predicate-based methods are expressive but incur substantial preprocessing and execution overhead, whereas interval-based methods execute efficiently but restrict expressiveness through a predefined interval count. We present CompileMask, which addresses this trade-off by connecting expressive mask specifications with efficient attention execution. We design a compiler that analyzes visibility rules during the offline stage and generates a mask-specific preprocess kernel. We further propose MaskSpec, a region-based DSL that lets users express these rules. During the online stage, the generated kernel reads batch metadata and produces a compact Execution Descriptor that lets the attention kernel efficiently check visibility by comparing element indices against precomputed endpoints. Experiments show that CompileMask achieves up to $3.8\times$ speedup over state-of-the-art methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑