面向自适应多臂老虎机的改进汤普森学习：数据中心可持续的节能灵活性调度
Refined Thompson Learning for Adaptive Bandits: Power-Efficient Flexibility Scheduling Across Data Centers
浏览论文内容
中文总结 AI 辅助
针对数据中心AI工作负载能耗给电网带来的压力，该研究提出融入领域知识的改进汤普森学习RMAB框架，模拟显示其在压力场景下优于TW且计算成本低于EXP4，凸显数据中心灵活性服务的经济价值及高质量AI数据集的重要性。
中文摘要 AI 辅助
近年来，数据中心大规模AI工作负载的能耗快速增长，给电网带来了日益增大的压力。由于电网必须实时维持供需平衡，数据中心的作业灵活性服务受到了越来越多的关注。我们提出了一种自适应的、基于学习的上下文 restless 多臂老虎机（RMAB）框架，在该框架中，电网请求负载削减，且无需知晓作业（重）调度决策。利用多个虚拟机（VM）数据集，核心策略将每个数据中心内的循环作业队列和批次级（重）调度建模为马尔可夫决策过程（MDP），并通过汤普森采样从学习到的转移和奖励函数中推导基于Whittle指数的策略。为解决状态空间扩大和状态访问稀疏的挑战，我们引入了融入领域知识的改进策略，包括自适应混合策略、门控先验、低秩平滑和离线后验支持。在基准和压力场景下的大量模拟表明，在压力扫描的每个报告单元中，表现最佳的改进变体在比EXP4更低的计算成本下优于TW。这些结果证明了数据中心灵活性服务的经济价值，并强调了高质量开源AI工作负载数据集对开发和评估自适应调度算法的重要性。
英文摘要
The rapid growth of large-scale AI workloads in data centers has placed increasing pressure on power grids in recent years. Since power systems must continuously balance supply and demand, there is growing interests in leveraging data-center workload flexibility as a grid service. We propose a contextual restless multi-armed bandit (CRMAB) framework in which a grid operator requests load reductions without observing internal job-scheduling decisions. Under index-ability guarantee, each data center or physical machine is modeled as a Markov decision process (MDP) over a cyclic virtual-machine (VM) job queue, with unknown rewards and transition dynamics learned online using Thompson sampling and Whittle-index policies. To improve learning under sparse and noisy observations, the framework augments an adaptive Thompson--Whittle (TW) policy with domain-informed transition priors and gated prior mixing. In baseline experiments, the best adaptive refined variant achieves 91.4\% of the oracle reward after 100 rounds and 96.8\% after 1,000 rounds. Across a 16-setting stress test spanning different state-space sizes and levels of contextual noise, the best refined variant consistently outperforms the original TW policy with high confidence while remaining competitive with EXP4. A graph-based prior further incorporates data-center hardware constraints, including computing-resource limits. Overall, the results demonstrate the economic potential of data-center flexibility as a grid service and highlight the importance of high-quality, open-source AI workload traces for developing and evaluating such services.
