发表机构
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究通过全栈能量剖析，揭示LLM智能体推理中GPU遥测遗漏大量系统能耗，并发现顺序执行能耗高、扩展思考增加能耗源于输出量，连续批处理可提升能效。
AI 中文摘要
LLM智能体通过迭代推理、规划和调用工具，产生了与单次推理根本不同的工作负载特征，然而其能量消耗在硬件组件和工作负载阶段之间的分布仍鲜为人知。因此，表征这些工作负载需要同时具备组件级功率和阶段级执行的可见性。我们开展了一项全栈能量剖析研究，结合NVML GPU计数器、Intel RAPL CPU/DRAM计数器以及智能平台管理接口（IPMI）系统级传感器，在2块NVIDIA RTX PRO 6000 Blackwell GPU上，使用Qwen3.8-27B对三个代表性工作负载进行了剖析。对于数学推理，我们比较了启用思考与禁用思考两种模式。我们的测量结果显示，仅依赖GPU遥测会遗漏所有三个工作负载中41-45%的系统能量，其余部分由非GPU组件消耗。在我们的设置中，顺序智能体工作负载每个输出token消耗的系统能量是饱和服务的63倍，这反映了缺乏批处理、跨轮次的上下文增长以及工具引发的空闲期。扩展思考使每个问题生成的token数量增加21-75%，而每种数据集内两种模式之间的每token能量差异小于1%，表明由此产生的能量增加是由输出量驱动的，而非每token效率的变化。连续批处理将系统级能量效率从每秒1个请求提高到每秒16个请求时提升了3.2倍，因为GPU功率趋于平稳，而吞吐量随批处理深度继续增加。对于受内存带宽限制的推理工作负载，在每GPU功率上限为400-600 W时吞吐量保持不变，但在300 W时急剧下降。这些发现表明，诸如摘要和选择性检索等上下文管理技术可能有助于降低智能体部署中的能量消耗。
英文摘要
LLM agents that iteratively reason, plan, and invoke tools create workload profiles fundamentally different from single-pass inference, yet how their energy consumption is distributed across hardware components and workload phases remains poorly understood. Characterizing these workloads therefore requires simultaneous visibility into both component-level power and phase-level execution. We conduct a full-stack energy profiling study combining NVML GPU counters, Intel RAPL CPU/DRAM counters, and Intelligent Platform Management Interface (IPMI) system-level sensors on 2x NVIDIA RTX PRO 6000 Blackwell GPUs, and profile three representative workloads with Qwen3.8-27B. For mathematical reasoning, we compare thinking-enabled and thinking-disabled modes. Our measurements reveal that GPU-only telemetry misses 41-45% of system energy across all three workloads, with non-GPU components accounting for the remainder. In our setup, the sequential agent workload consumes 63x more system energy per output token than saturated serving, reflecting the absence of batching, context growth across turns, and tool-induced idle periods. Extended thinking generates 21-75% more tokens per problem, while per-token energy differs by less than 1% between modes within each dataset, indicating that the resulting increase in energy is driven by output volume rather than a change in per-token efficiency. Continuous batching improves system-level energy efficiency by 3.2x from 1 to 16 requests per second, as GPU power plateaus while throughput continues to increase with batching depth. For the memory-bandwidth-bound reasoning workload, throughput remains unchanged under per-GPU power caps of 400-600 W but drops sharply at 300 W. These findings suggest that context-management techniques such as summarization and selective retrieval may help reduce energy consumption in agent deployments.
Comments11 pages, 6 figures
