arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.05904cs.ARcs.AI
收藏

LUMINA: 通过瓶颈分析指导的LLM引导的GPU架构探索

LUMINA: LLM-Guided GPU Architecture Exploration via Bottleneck Analysis

Microsoft Research(微软研究院)

Tao Zhang, Rui Ma, Shuotao Xu, Yongqiang Xiong, Peng Cheng

AI总结:

LUMINA利用AI提升GPU设计空间探索效率,通过自动编译DSE规则和基准测试,实现高效高质量的设计指导,比传统方法提升17.5倍效率和32.9%的设计效果。

摘要:

现代AI工作负载(如大语言模型推理)的GPU设计空间探索(DSE)极具挑战性,由于GPU设计空间庞大、多模态、模拟成本高且优化目标复杂(如性能、功耗和面积的权衡)。现有自动化DSE方法往往成本过高,要么需要过多探索样本,要么依赖人工编写的依赖路径分析。我们提出LUMINA,一个基于LLM的GPU架构探索框架,利用AI提升DSE效率和效果。LUMINA从模拟器代码中提取架构知识并进行敏感性研究,自动编译DSE规则,这些规则在探索过程中自动纠正。LUMINA的核心组件是一个DSE基准,全面评估和增强LLM在架构优化所需三项基本技能上的能力,为模型选择提供原理性和可重复的基础,并确保一致的架构推理。在包含470万可能样本的设计空间中,LUMINA通过LLM辅助瓶颈分析仅用20步就高效识别出6种性能和面积优于A100 GPU的设计。相比传统方法,LUMINA在设计空间探索效率上提升17.5倍,并在Pareto超体积上比机器学习基线高出32.9%,展示了其在最小搜索成本下提供高质量设计指导的能力。

英文摘要:

GPU design space exploration (DSE) for modern AI workloads, such as Large-Language Model (LLM) inference, is challenging because of GPUs' vast, multi-modal design spaces, high simulation costs, and complex design optimization objectives (e.g. performance, power and area trade-offs). Existing automated DSE methods are often prohibitively expensive, either requiring an excessive number of exploration samples or depending on intricate, manually crafted analyses of interdependent critical paths guided by human heuristics. We present LUMINA, an LLM-driven GPU architecture exploration framework that leverage AI to enhance the DSE efficiency and efficacy for GPUs. LUMINA extracts architectural knowledge from simulator code and performs sensitivity studies to automatically compose DSE rules,which are auto-corrected during exploration. A core component of LUMINA is a DSE Benchmark that comprehensively evaluates and enhances LLMs' capabilities across three fundamental skills required for architecture optimization, which provides a principled and reproducible basis for model selection and ensuring consistent architectural reasoning. In the design space with 4.7 million possible samples, LUMINA identifies 6 designs of better performance and area than an A100 GPU efficiently, using only 20 steps via LLM-assisted bottleneck analysis. In comparison, LUMINA achieves 17.5x higher than design space exploration efficiency, and 32.9% better designs (i.e. Pareto Hypervolume) than Machine-Learning baselines, showcasing its ability to deliver high-quality design guidance with minimal search cost.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑