arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.12698cs.CL
收藏

一种结合语义重构和基于搜索的优化的双阶段GPU内核调优器

A Two-Stage GPU Kernel Tuner Combining Semantic Refactoring and Search-Based Optimization

NanKai University(南开大学);Beijing Institute of Computer Technology and Applications(北京计算机技术与应用研究所)

Qiuyi Qu, Yicheng Sui, Yufei Sun, Rui Chen, Xiaofei Zhang, Yuzhi Zhang, Haofeng Wang, Ge Lan

AI总结:

本文提出一种结合语义重构和基于搜索的双阶段GPU内核调优器,通过模板化和参数优化提升内核性能,实现更稳定和高质量的加速效果。

摘要:

GPU代码优化是HPC工作负载以及大模型训练和推理中的关键性能瓶颈。尽管编译器优化和手写内核可以部分缓解这一问题,但实现接近硬件极限的性能仍然严重依赖于手动代码重构和参数调优。最近在基于LLM代理的内核生成和优化方面取得了一些进展,但许多方法主要集中在直接代码重写上,其中参数选择往往隐含且难以控制,或者需要人工干预,导致性能提升不稳定。本文在代理驱动的迭代循环上引入了一层基于模板的重写层:内核被语义重构为显式参数化的模板,然后通过基于搜索的自动调优优化模板参数,从而获得更稳定和更高质量的加速效果。在一组真实世界的内核上进行的实验显示,最佳情况下加速超过了3倍。我们从SGLang中提取了具有代表性的CUDA内核作为评估目标;所提出的代理调优器会迭代执行模板化、测试、分析和规划,并利用性能剖析反馈在硬件资源限制下执行受约束的参数搜索。与仅基于代理的直接重写相比,模板加搜索的设计显著减少了迭代优化的随机性,使过程更加可解释,并使朝着高性能配置的系统方法更加可行。所提出的方法可以进一步扩展到OpenCL、HIP和其他后端,以实现对真实生产工作负载的自动化性能优化。

英文摘要:

GPU code optimization is a key performance bottleneck for HPC workloads as well as large-model training and inference. Although compiler optimizations and hand-written kernels can partially alleviate this issue, achieving near-hardware-limit performance still relies heavily on manual code refactoring and parameter tuning. Recent progress in LLM-agent-based kernel generation and optimization has been reported, yet many approaches primarily focus on direct code rewriting, where parameter choices are often implicit and hard to control, or require human intervention, leading to unstable performance gains. This paper introduces a template-based rewriting layer on top of an agent-driven iterative loop: kernels are semantically refactored into explicitly parameterizable templates, and template parameters are then optimized via search-based autotuning, yielding more stable and higher-quality speedups. Experiments on a set of real-world kernels demonstrate speedups exceeding 3x in the best case. We extract representative CUDA kernels from SGLang as evaluation targets; the proposed agentic tuner iteratively performs templating, testing, analysis, and planning, and leverages profiling feedback to execute constrained parameter search under hardware resource limits. Compared to agent-only direct rewriting, the template-plus-search design significantly reduces the randomness of iterative optimization, making the process more interpretable and enabling a more systematic approach toward high-performance configurations. The proposed method can be further extended to OpenCL, HIP, and other backends to deliver automated performance optimization for real production workloads.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑