发表机构
HKUST; Tsinghua University（香港科技大学; 清华大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对电力采购承诺偏差成本问题，提出ePACT两级控制器，通过调整服务容量与GPU时钟跟踪小时级能量承诺，在保持SLO的同时显著降低不对称偏差成本。
AI 中文摘要
减少LLM服务能耗本身并不能保证降低部署成本，因为电力采购会使运营商面临偏离预设承诺的不利情况。我们研究了小时级承诺，其中超用和欠用具有正的、可能不对称的成本，并提出了能量-性能感知承诺跟踪问题：在满足请求级服务要求的前提下，最小化偏差成本。我们实现了ePACT，一个两级控制器，在请求到达时调整服务容量和GPU时钟。全局规划器根据实测消耗和剩余的小时承诺更新区间能量目标。本地决策器预测候选配置的能量和完成时间，检查预测的截止时间错过情况，并根据不对称目标偏差成本（具有服务优先回退）在已接纳的配置中进行选择。粗到细的动作搜索与服务异步运行。我们通过单小时比较、控制器消融以及H20和H200 GPU池的全天轨迹模拟来评估ePACT。在24小时模拟中，相对于vLLM，ePACT将不对称偏差成本降低了73.8%和75.7%，同时保持了接近vLLM的SLO达成率。平均绝对小时偏差分别为2.16%和2.31%。
英文摘要
Reducing LLM serving energy does not by itself guarantee lower deployment cost when electricity procurement exposes operators to unfavorable deviations from preset commitments. We study hourly commitments with positive, potentially asymmetric costs for overuse and underuse, and formulate energy-Performance-Aware Commitment Tracking: minimize deviation costs subject to request-level service requirements. We implement ePACT, a two-level controller that adjusts serving capacity and GPU clocks as requests arrive. A global planner updates interval energy targets from measured consumption and the remaining hourly commitment. A local decision maker predicts candidate configurations' energy and completion times, checks predicted deadline misses, and selects among admitted configurations by asymmetric target-deviation cost, with a service-first fallback. Coarse-to-fine action search runs asynchronously with serving. We evaluate ePACT through single-hour comparisons, controller ablations, and full-day trace simulations for H20 and H200 GPU pools. In the 24-hour simulations, ePACT reduces the asymmetric deviation cost by $73.8\%$ and $75.7\%$ relative to vLLM while retaining near-vLLM SLO attainment. Mean absolute hourly deviations are $2.16\%$ and $2.31\%$, respectively.
