arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.05695cs.AIcs.PF
收藏

SweetSpot: 一种预测LLM推理能效的分析模型

SweetSpot: An Analytical Model for Predicting Energy Efficiency of LLM Inference

University of Bologna(博洛尼亚大学)

Hiari Pizzini Cavagna, Andrea Proia, Giacomo Madella, Giovanni B. Esposito, Francesco Antici, Daniele Cesarini, Zeynep Kiziltan, Andrea Bartolini

AI总结:

SweetSpot通过分析Transformer架构,提出预测LLM推理能效的模型,通过实验验证其在不同输入输出长度下的高效能表现,减少能耗达33.41倍。

摘要:

大型语言模型(LLM)的推理是现代人工智能应用的核心,主导着全球数据中心的工作负载,因此预测其能耗至关重要。现有方法将能耗估计为输入和输出序列长度的简单线性函数。然而,通过分析Transformer的自回归结构,我们发现输入和输出序列长度与能耗之间存在根本性的非线性关系,从而证明了生成能耗最小值的存在。峰值效率出现在短至中等长度的输入和中等长度的输出,而长输入或极短输出会导致效率急剧下降。因此,我们提出了SweetSpot,一种基于Transformer架构计算和内存访问复杂度的分析模型,能够准确地将效率曲线表示为输入和输出长度的函数。为了评估准确性,我们使用TensorRT-LLM在NVIDIA H100 GPU上测量了不同参数范围(从1B到9B)的LLM的能耗,包括OPT、LLaMA、Gemma、Falcon、Qwen2和Granite。我们测试了输入和输出长度从64到4096个标记,并实现了平均MAPE为1.79%。我们的结果表明,将序列长度与这些效率“甜点”对齐可以减少能耗,最高可达33.41倍,从而实现有意识的截断、摘要和自适应生成策略,在生产系统中具有指导意义。

英文摘要:

Large Language Models (LLMs) inference is central to modern AI applications, dominating worldwide datacenter workloads, making it critical to predict its energy footprint. Existing approaches estimate energy consumption as a simple linear function of input and output sequence. However, by analyzing the autoregressive structure of Transformers, which implies a fundamentally non-linear relationship between input and output sequence lengths and energy consumption, we demonstrate the existence of a generation energy minima. Peak efficiency occurs with short-to-moderate inputs and medium-length outputs, while efficiency drops sharply for long inputs or very short outputs. Consequently, we propose SweetSpot, an analytical model derived from the computational and memory-access complexity of the Transformer architecture, which accurately characterizes the efficiency curve as a function of input and output lengths. To assess accuracy, we measure energy consumption using TensorRT-LLM on NVIDIA H100 GPUs across a diverse set of LLMs ranging from 1B to 9B parameters, including OPT, LLaMA, Gemma, Falcon, Qwen2, and Granite. We test input and output lengths from 64 to 4096 tokens and achieve a mean MAPE of 1.79%. Our results show that aligning sequence lengths with these efficiency "sweet spots" reduce energy usage, up to 33.41x, enabling informed truncation, summarization, and adaptive generation strategies in production systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑