arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.18755cs.DC
收藏

DualScale:通过相意识放置和DVFS实现能量高效的解耦LLM服务

DualScale: Energy-Efficient Disaggregated LLM Serving via Phase-Aware Placement and DVFS

Purdue University(普渡大学)

Omar Basit, Yunzhao Liu, Z. Jonny Kong, Y. Charlie Hu

AI总结:

DualScale通过预测延迟和功率模型,在预填和解码阶段联合优化放置和DVFS,以满足SLO约束并减少能耗,实验显示在预填和解码中分别降低能耗39%和48%。

摘要:

DualScale通过预测延迟和功率模型,在预填和解码阶段联合优化放置和DVFS,以满足SLO约束并减少能耗,实验显示在预填和解码中分别降低能耗39%和48%。

英文摘要:

Prefill/decode disaggregation is increasingly adopted in LLM serving to improve the latency-throughput tradeoff and meet strict TTFT and TPOT SLOs. However, LLM inference remains energy-hungry: autoscaling alone is too coarse-grained to track fast workload fluctuations, and applying fine-grained DVFS under disaggregation is complicated by phase-asymmetric dynamics and coupling between provisioning and frequency control. We present DualScale, a two-tier energy optimization framework for disaggregated LLM serving. DualScale jointly optimizes placement and DVFS across prefill and decode using predictive latency and power models. At coarse timescales, DualScale computes phase-aware placement and baseline frequencies that minimize energy while satisfying SLO constraints. At fine timescales, DualScale dynamically adapts GPU frequency per iteration using stage-specific control: model predictive control (MPC) for prefill to account for queue evolution and future TTFT impact, and lightweight slack-aware adaptation for decode to exploit its smoother, memory-bound dynamics. This hierarchical design enables coordinated control across timescales while preserving strict serving SLOs. Evaluation on a 16x H100 cluster serving Llama 3.3 70B with production-style traces shows that DualScale meets TTFT/TPOT SLOs while reducing energy by up to 39% in prefill and 48% in decode relative to DistServe.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑