arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2611.03330cs.AIcs.LGcs.PF
收藏

LLM推理的SIMPLE能量成本定律

The SIMPLE Law for Energy Cost of LLM Inference

University of Bologna(博洛尼亚大学);CINECA(奇内卡国家计算中心)

Hiari Pizzini Cavagna, Andrea Proia, Daniele Cesarini, Zeynep Kiziltan, Andrea Bartolini

AI总结:

本研究提出SIMPLE定律,推导LLM推理的计算与内存复杂度闭式表达式,构建四参数SIMPLE模型,在H100 GPU上对13个LLM实现高预测精度,揭示GQA分组、推理预算等对能量成本的影响。

摘要:

生成式AI按token计费,但不同token的成本并不相同。每个token都会进入KV缓存,并提高后续token的成本,因此当序列中的token相同时,能量会沿序列非线性增长。目前没有分析模型能明确token的成本,最准确的SOTA模型是拟合模型而非定律:它将架构信息融入拟合系数,需要针对每个LLM单独拟合。我们推导了LLM推理的计算和内存访问复杂度的闭式表达式,涵盖MHA和GQA,并比较了一系列粒度递增的分析模型。由此得到SIMPLE推理模型(全称是Predict LLMs Energy的SIMPLE),这是一个四参数定律,将prefill(预填充)与decode(解码)分开,并将解码分为KV缓存成本和下一个token计算成本。在NVIDIA H100 GPU上,针对6个系列的13个密集LLM的轨迹,该模型达到R²=0.969和11.97%的MAPE,比SOTA低3.1-3.5倍,且无需针对每个LLM校准。SIMPLE定律兼具解释性和预测性:预填充能量随隐藏维度二次增长,因此更大的LLM在处理长输入时受影响最严重;更激进的GQA KV分组相对于标准MHA可将每个token的能量降低多达3倍。增加推理token预算会使输出长度超过分析得出的能量最小值,推动能量超线性增长;尽管准确率随预算提高,但能量增长更快,因此每焦耳的智能在中间预算达到峰值,在4.0倍推理预算范围内下降多达3.14倍。智能体服务相对于非智能体服务的每个请求能量提高13.0倍,原因是预填充和KV缓存项随输入长度增长;而前缀缓存可消除大部分预填充成本(-93%),但对KV缓存项无影响,因此即使是缓存后的智能体请求,成本仍高出4.8倍。

英文摘要:

Generative AI is billed per token, yet tokens are not equally expensive. Each token enters the KV-cache and raises the cost of those that follow, so energy grows non-linearly along a sequence whose tokens look identical. No analytical model states what a token costs without per-LLM fitting, and no law names the factors that set it. From closed-form expressions of LLM inference's computational and memory-access complexity, we derive the SIMPLE Inference Model to Predict LLMs Energy (SIMPLE), a four-parameter law that separates prefill from decode, and decode into KV-cache and next-token computation cost. Over traces of 13 dense LLMs from several families on NVIDIA H100 GPUs, it attains R2 = 0.969 and 11.97% MAPE, 3.1-3.5x below the SOTA, without per-LLM calibration. SIMPLE explains as well as predicts: prefill energy grows quadratically with hidden dimension, so larger LLMs suffer most from long inputs; and higher GQA KV grouping reduces the per-token energy linearly relative to standard MHA. Increasing the reasoning token budget pushes output length beyond the analytically derived energy minimum, driving super-linear energy growth; accuracy rises but energy rises faster, so Intelligence per Joule peaks at an intermediate budget and falls by up to 3.14x across a 4.0x range. Moving to agentic serving raises energy per request by 13.0x, as both prefill and KV-cache terms grow with input length; prefix caching removes most of the prefill cost (-93%), but not KV-cache cost, so even a cached agentic request costs 4.8x more.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑