面向可再生能源主导的AI数据中心的大语言模型预训练负载的执行可行调度
Execution-Feasible Dispatch of LLM Pre-training Loads for Renewable-Dominant AI Data Centers
AI总结:
该研究针对可再生能源主导的AI数据中心,提出感知检查点的三维并行GPU调度框架,可降低LLM预训练的运营成本并减少可再生能源弃电量。
摘要:
AI数据中心(AIDC)正成为规模庞大的电力系统负载,其运行灵活性的价值日益凸显。现有研究常将AIDC的需求视为可连续调节或普遍可延迟的,这高估了大语言模型(LLM)预训练的灵活性。本文提出一种面向可再生能源主导的AIDC中LLM预训练的执行可行调度框架:首先将LLM描述符转换为检查点域的每日工作负载,再映射到具有明确吞吐量、GPU容量、内存和IT功率限制的可行三维并行GPU模式;将得到的检查点和模式约束的负载状态嵌入日前电力调度模型。对拥有2048个GPU的AIDC开展两项LLaMA预训练任务的案例研究表明,与固定连续训练相比,感知检查点的三维调度可将运营成本降低20.02%,可再生能源弃电量从1.077 MWh降至0.014 MWh;相关结果还显示,仅检查点记录无法产生负载灵活性,实际的电网交互灵活性源于检查点等待与执行可行三维模式补偿的结合。
英文摘要:
AI data centers (AIDCs) are emerging as large power-system loads, making their operational flexibility increasingly valuable. Existing studies often treat AIDC demand as continuously adjustable or generically delay-tolerant, which overstating the flexibility of large language model (LLM) pretraining. This paper proposes an execution-feasible dispatch framework for LLM pre-training in renewable-dominant AIDCs. LLM descriptors are first converted into checkpoint-domain daily workloads, then mapped to feasible 3D-parallel GPU modes with explicit throughput, GPU-capacity, memory, and IT-power limits. The resulting checkpoint- and mode-constrained load states are embedded into a day-ahead power dispatch model. Case studies on a 2048 GPUs AIDC with two LLaMA pretraining tasks show that checkpoint-aware 3D dispatch reduces operating cost by 20.02% and renewable curtailment from 1.077 to 0.014 MWh relative to fixed continuous training. Related results also show that checkpoint records alone do not create load flexibility,practical grid-interactive flexibility arises from checkpoint waiting combined with execution-feasible 3D-mode compensation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。