arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.01489cs.LGcs.AIcs.DCcs.PFcs.SE
收藏

CuTeGen: 基于LLM的代理框架用于使用CuTe生成和优化高性能GPU内核

CuTeGen: An LLM-Based Agentic Framework for Generation and Optimization of High-Performance GPU Kernels using CuTe

Department of Computer Science, University of Toronto(计算机科学系,多伦多大学)

Tara Saba, Zhiyang Chen, Jikai Jason Li, Anne Ouyang, Xujie Si, Fan Long

AI总结:

本文提出CuTeGen,一种基于LLM的代理框架,通过CuTe抽象层实现GPU内核的生成和优化,通过结构化生成-测试-优化工作流,在标准基准测试中实现了比PyTorch快1.71倍的速度提升,并在生成成本相近的情况下优于现有代理基线CudaForge。

摘要:

高性能GPU内核对现代机器学习系统至关重要,但开发这些内核仍然是一个手动、专家驱动的过程。最近的研究尝试利用LLM自动生成功能内核,但生成的内核在标准化基准测试中仍无法达到精心调优的参考内核。我们提出了CuTeGen,一种代理GPU内核合成框架,将内核开发视为在CuTe抽象层上的结构化生成-测试-优化工作流。CuTeGen有两个设计选择区别于先前的工作:针对CuTe而不是原始CUDA,这暴露了性能关键结构如分块和数据移动,同时保持足够的稳定性以进行迭代优化;以及延迟的性能调度,将低层次性能反馈推迟到内核的高层结构稳定之后。在209个KernelBench Level-1和Level-2任务上,CuTeGen在PyTorch上实现了平均1.71倍的速度提升,并在生成成本相近的情况下优于先前的代理基线CudaForge(0.89倍)。代码可在https://github.com/taratt/cutegen.git获取。

英文摘要:

High-performance GPU kernels are critical to modern machine learning systems, yet developing them remains a manual, expert-driven process. Recent work has explored using LLMs to automate kernel generation, but generated kernels still fall short of carefully tuned references on standardized benchmarks. We present CuTeGen, an agentic GPU kernel synthesis framework that treats kernel development as a structured generate-test-refine workflow over the CuTe abstraction layer. Two design choices distinguish CuTeGen from prior work: targeting CuTe rather than raw CUDA, which exposes performance-critical structures such as tiling and data movement while remaining stable enough for iterative refinement, and a delayed profiling schedule that withholds low-level performance feedback until the kernel's high-level structure has stabilized. On the 209 tasks of KernelBench Level-1 and Level-2, CuTeGen achieves an average speedup of 1.71$\times$ over PyTorch and outperforms the prior agentic baseline CudaForge (0.89$\times$) at comparable per-task generation cost. Code available at https://github.com/taratt/cutegen.git

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑