Aryl：面向深度学习的弹性集群调度器
Aryl: An Elastic Cluster Scheduler for Deep Learning
AI总结：
Aryl通过容量借用和弹性伸缩优化深度学习集群调度，减少排队时间和作业完成时间，提升集群利用率。
摘要：
企业为深度学习分别构建训练和推理GPU集群，并使用独立的调度器进行管理。这给训练和推理都带来了问题：推理集群在流量负载低时GPU利用率低；训练作业常因资源不足而经历长时间排队。我们引入Aryl，一种新的集群调度器来解决这些问题。Aryl引入容量借用机制，将空闲的推理GPU服务器借用给训练作业。它进一步利用弹性伸缩，调整训练作业的GPU分配，以更好地利用借用的资源。容量借用和弹性伸缩给集群管理带来了新的挑战。当需要归还借用的服务器时，我们需要最小化作业抢占的数量；当有更多GPU可用时，我们需要将它们分配给弹性作业，并最小化作业完成时间（JCT）。Aryl使用有原则的启发式方法解决这些组合问题。它引入了服务器抢占成本的概念，并在服务器回收过程中贪婪地降低该成本。它进一步依赖于为弹性作业的每个额外工作线程定义的JCT减少值，将调度问题作为多选背包问题来解决。在64-GPU测试平台上的原型实现和包含超过50,000个生产作业的15天轨迹的大规模模拟表明，与没有容量借用或弹性伸缩的集群调度器相比，Aryl将平均排队时间和JCT分别降低了1.53倍和1.50倍，并将集群利用率提高了高达26.9%。
英文摘要：
Companies build separate training and inference GPU clusters for deep learning, and use separate schedulers to manage them. This leads to problems for both training and inference: inference clusters have low GPU utilization when the traffic load is low; training jobs often experience long queueing time due to lack of resources. We introduce Aryl, a new cluster scheduler to address these problems. Aryl introduces capacity loaning to loan idle inference GPU servers for training jobs. It further exploits elastic scaling that scales a training job's GPU allocation to better utilize loaned resources. Capacity loaning and elastic scaling create new challenges to cluster management. When the loaned servers need to be returned, we need to minimize the number of job preemptions; when more GPUs become available, we need to allocate them to elastic jobs and minimize the job completion time (JCT). Aryl addresses these combinatorial problems using principled heuristics. It introduces the notion of server preemption cost which it greedily reduces during server reclaiming. It further relies on the JCT reduction value defined for each additional worker for an elastic job to solve the scheduling problem as a multiple-choice knapsack problem. Prototype implementation on a 64-GPU testbed and large-scale simulation with 15-day traces of over 50,000 production jobs show that Aryl brings 1.53x and 1.50x reductions in average queuing time and JCT, and improves cluster usage by up to 26.9% over the cluster scheduler without capacity loaning or elastic scaling.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
