面向虚拟机调度的可扩展强化学习
Scalable Reinforcement Learning for Virtual Machine Scheduling
AI总结：
本文提出可扩展强化学习框架CVD-RL，通过结合分解算子、前向观察算子与Top-$k$过滤算子解决大规模虚拟机调度的可扩展性难题，将适用规模扩展至50台物理机，并在多种场景下泛化能力超越SOTA方法。
摘要：
近期强化学习（RL）的进展表明其在小规模集群中优化虚拟机调度（VMS）具有潜力。然而，将RL应用于大规模云计算场景仍受到显著限制。本文提出了一种名为Cluster Value Decomposition Reinforcement Learning (CVD-RL)的可扩展RL框架，以克服大规模VMS固有的可扩展性障碍。CVD-RL框架创新性地将分解算子与前向观察算子相结合，以熟练地处理表示复杂性，同时辅以Top-$k$过滤算子来提升探索效率。与现有方法仅限于$10$台或更少物理机（PMs）的集群不同，CVD-RL将其适用范围扩展至包含多达$50$台PMs的环境。此外，在实证研究中，CVD-RL框架展现出的泛化能力在多种场景下超越了当代SOTA方法。这一突破不仅展示了该框架卓越的可扩展性和性能，也标志着RL在复杂、大规模云基础设施VMS应用中的重大飞跃。代码可在https://anonymous.4open.science/r/marl4sche-D0FE获取。
英文摘要：
Recent advancements in reinforcement learning (RL) have shown promise for optimizing virtual machine scheduling (VMS) in small-scale clusters. The utilization of RL to large-scale cloud computing scenarios remains notably constrained. This paper introduces a scalable RL framework, called Cluster Value Decomposition Reinforcement Learning (CVD-RL), to surmount the scalability hurdles inherent in large-scale VMS. The CVD-RL framework innovatively combines a decomposition operator with a look-ahead operator to adeptly manage representation complexities, while complemented by a Top-$k$ filter operator that refines exploration efficiency. Different from existing approaches limited to clusters of $10$ or fewer physical machines (PMs), CVD-RL extends its applicability to environments encompassing up to $50$ PMs. Furthermore, the CVD-RL framework demonstrates generalization capabilities that surpass contemporary SOTA methodologies across a variety of scenarios in empirical studies. This breakthrough not only showcases the framework's exceptional scalability and performance but also represents a significant leap in the application of RL for VMS within complex, large-scale cloud infrastructures. The code is available at https://anonymous.4open.science/r/marl4sche-D0FE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
