BOA Constrictor: 通过预算最优分配从云中挤出GPU性能
BOA Constrictor: Squeezing Performance out of GPUs in the Cloud via Budget-Optimal Allocation
AI总结：
BOA Constrictor通过预算最优分配策略优化云中GPU资源分配，减少训练任务的平均完成时间。
摘要：
过去十年中，GPU对训练机器学习（ML）模型的需求急剧增加。由于大多数组织无法承担构建和维护大型GPU集群的成本，组织倾向于从云服务提供商租用GPU。客户需要制定策略来决定何时租用多少GPU来处理流式ML训练任务，并将租用的GPU分配给当前活跃的任务。由于ML训练任务可以在不同数量的GPU上并行化，客户通常有多种选择来决定每个任务使用多少GPU。为单个训练任务分配更多GPU会使其完成得更快。然而，客户为每个GPU小时付费，而训练任务从额外GPU上运行获得的边际收益会递减。因此，为单个训练任务分配过多GPU会显著增加客户支付给云服务提供商的总体成本。这导致了客户在云中运行训练任务时必须平衡的成本性能权衡。为了平衡这种成本性能权衡，我们开发了BOA Constrictor，这是一种新的ML训练任务调度器，使用预算最优分配（BOA）策略，在给定固定预算约束下，从云部署的GPU集群中挤出最高水平的性能。我们明确将问题表述为一个预算约束的调度问题，并推导出BOA策略，该策略在用户预算限制下最小化一系列到达任务的平均作业完成时间（JCT）。对于给定的预算水平，我们证明BOA Constrictor在小型规模实现实验中可以将平均JCT减少1.6倍，在详细的大规模模拟中相比最先进的启发式调度器减少2倍。
英文摘要：
The past decade has seen a dramatic increase in demand for GPUs to train Machine Learning (ML) models. Because it is prohibitively expensive for most organizations to build and maintain a large GPU cluster, organizations instead choose to rent GPUs from cloud providers. The customer is responsible for devising a policy for (i) deciding how many GPUs to rent at every moment in time to process a stream of ML training jobs and (ii) allocating the rented GPUs among the currently active jobs in the system. Because ML training jobs can be parallelized across different numbers of GPUs, the customer generally has many options for how many GPUs to use for each job. Allocating more GPUs to a single training job will cause the job to complete more quickly. However, the customer pays for each GPU-hour they use, and a training job receives a diminishing marginal benefit from running on additional GPUs. Hence, allocating too many GPUs to a single training job can dramatically increase the overall cost that the customer pays to the cloud provider. This gives rise to a cost-performance tradeoff that customers must balance when running training jobs in the cloud. To balance the cost-performance tradeoff, we develop BOA Constrictor, a new scheduler for ML training jobs which uses a Budget-Optimal Allocation (BOA) policy to squeeze the highest level of performance out of a cloud-deployed GPU cluster given a fixed budget constraint. We explicitly formulate the problem as a budget-constrained scheduling problem and derive the BOA policy which minimizes the average job completion time (JCT) of a stream of arriving jobs subject to the user's budget. For a given budget level, we demonstrate that BOA Constrictor can reduce average JCT by 1.6 times in small-scale implementation experiments and by 2 times in detailed, large-scale simulations compared to state-of-the-art heuristic based schedulers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
