arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.03024cs.LGcs.AIcs.CYcs.DC
收藏

TokenPowerBench:评估大语言模型推理的能耗基准

TokenPowerBench: Benchmarking the Power Consumption of LLM Inference

Chenxu Niu, Wei Zhang, Jie Li, Yongjian Zhao, Tongyang Wang, Xi Wang, Yong Chen

AI总结:

TokenPowerBench是首个用于评估大语言模型推理能耗的轻量可扩展基准,通过测量和分析不同设置对能效指标的影响,帮助用户优化能耗和可持续性。

摘要:

大语言模型(LLM)服务现在每天回答数十亿个查询,行业报告表明,推理,而非训练,占总能耗的90%以上。然而,现有基准主要关注训练/微调或推理性能,对推理能耗的测量和分析支持有限。我们引入TokenPowerBench,首个轻量且可扩展的基准,用于LLM推理能耗研究。该基准结合(i)声明式配置接口,涵盖模型选择、提示集和推理引擎;(ii)测量层,无需专用功耗仪即可捕获GPU、节点和系统级功耗;(iii)相位对齐的度量管道,将能量分配到每个请求的prefill和decode阶段。这些元素使探索LLM推理运行的能耗变得简单;此外,通过改变批处理大小、上下文长度、并行策略和量化,用户可以快速评估每个设置对每词焦耳数和其他能效指标的影响。我们在四个最广泛使用的模型系列(Llama、Falcon、Qwen和Mistral)上评估TokenPowerBench。实验涵盖从十亿参数到前沿规模的Llama3-405B模型。此外,我们开源TokenPowerBench,帮助用户测量能耗、预测运营成本,并满足部署LLM服务时的可持续性目标。

英文摘要:

Large language model (LLM) services now answer billions of queries per day, and industry reports show that inference, not training, accounts for more than 90% of total power consumption. However, existing benchmarks focus on either training/fine-tuning or performance of inference and provide little support for power consumption measurement and analysis of inference. We introduce TokenPowerBench, the first lightweight and extensible benchmark designed for LLM-inference power consumption studies. The benchmark combines (i) a declarative configuration interface covering model choice, prompt set, and inference engine, (ii) a measurement layer that captures GPU-, node-, and system-level power without specialized power meters, and (iii) a phase-aligned metrics pipeline that attributes energy to the prefill and decode stages of every request. These elements make it straight-forward to explore the power consumed by an LLM inference run; furthermore, by varying batch size, context length, parallelism strategy and quantization, users can quickly assess how each setting affects joules per token and other energy-efficiency metrics. We evaluate TokenPowerBench on four of the most widely used model series (Llama, Falcon, Qwen, and Mistral). Our experiments cover from 1 billion parameters up to the frontier-scale Llama3-405B model. Furthermore, we release TokenPowerBench as open source to help users to measure power consumption, forecast operating expenses, and meet sustainability targets when deploying LLM services.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑