PerfDojo:面向异构架构的自动化机器学习库生成
PerfDojo: Automated ML Library Generation for Heterogeneous Architectures
AI总结:
本文提出PerfLLM方法,通过PerfDojo环境将优化建模为强化学习博弈,利用人类可读代码表示和LLM实现跨异构CPU与GPU架构的自动性能优化。
摘要:
机器学习模型日益复杂,且多样化硬件架构(CPU、GPU、加速器)不断增多,使得实现最优性能成为一项重大挑战。指令集异构性、针对不同数据类型和模型特性(如稀疏性、量化)的专用内核需求,以及架构特定优化,都使性能调优变得更加复杂。手动优化资源消耗巨大,而现有自动方法往往依赖复杂的硬件特定启发式规则和难以解释的中间表示,阻碍了性能可移植性。我们提出PerfLLM,一种利用大型语言模型(LLMs)和强化学习(RL)的新型自动优化方法。其核心是PerfDojo,一个将优化构建为强化学习博弈的环境,使用人类可读、受数学启发的代码表示,通过变换保证语义有效性。这使得无需先验硬件知识即可进行有效优化,同时便于人类分析和强化学习智能体训练。我们展示了PerfLLM在多种CPU(x86、Arm、RISC-V)和GPU架构上取得显著性能提升的能力。
英文摘要:
The increasing complexity of machine learning models and the proliferation of diverse hardware architectures (CPUs, GPUs, accelerators) make achieving optimal performance a significant challenge. Heterogeneity in instruction sets, specialized kernel requirements for different data types and model features (e.g., sparsity, quantization), and architecture-specific optimizations complicate performance tuning. Manual optimization is resource-intensive, while existing automatic approaches often rely on complex hardware-specific heuristics and uninterpretable intermediate representations, hindering performance portability. We introduce PerfLLM, a novel automatic optimization methodology leveraging Large Language Models (LLMs) and Reinforcement Learning (RL). Central to this is PerfDojo, an environment framing optimization as an RL game using a human-readable, mathematically-inspired code representation that guarantees semantic validity through transformations. This allows effective optimization without prior hardware knowledge, facilitating both human analysis and RL agent training. We demonstrate PerfLLM's ability to achieve significant performance gains across diverse CPU (x86, Arm, RISC-V) and GPU architectures.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。