如何在预算内租用 GPU
How to Rent GPUs on a Budget
AI总结:
本文研究在长期预算约束下如何为机器学习训练作业流租用GPU,通过权衡训练成本与平均响应时间,推导出几乎不依赖作业到达与大小分布假设的最优GPU租用与分配策略。
摘要:
过去十年间,机器学习(ML)的爆发导致对用于训练 ML 模型的 GPU 需求急剧增加。由于对大多数用户而言,构建和维护大型 GPU 集群的成本过高,大型云提供商(Microsoft Azure、Amazon AWS、Google Cloud)在租用基于云的 GPU 需求上经历了爆炸性增长。在这种云计算范式中,用户必须指定其在每个时刻对 GPU 的需求,并为使用的每个 GPU 小时付费。已知 ML 训练作业具有不同程度的可并行性。给定一个 ML 训练作业流,用户通常希望最小化所有作业的平均响应时间。这里,作业的响应时间是指从作业到达至其完成的时间。此外,用户受限于一定的运营预算。具体而言,本文中用户被限制在长期时间平均值上每小时使用不超过 $b$ 个 GPU。问题在于如何在满足预算约束的同时最小化平均响应时间。由于训练作业从运行在额外的 GPU 上获得的边际效益递减,为单个训练作业分配过多 GPU 会显著增加用户支付的总成本。因此,最优租用策略必须在训练成本和平均响应时间之间权衡。本文推导了训练作业流的最优租用策略,其中作业具有不同的可并行性水平(由加速函数指定)和不同的作业大小(固有工作量)。我们几乎不对到达过程和作业大小分布做任何假设。我们的最优策略指定了在每个时刻租用多少 GPU 以及如何分配这些 GPU。
英文摘要:
The explosion in Machine Learning (ML) over the past ten years has led to a dramatic increase in demand for GPUs to train ML models. Because it is prohibitively expensive for most users to build and maintain a large GPU cluster, large cloud providers (Microsoft Azure, Amazon AWS, Google Cloud) have seen explosive growth in demand for renting cloud-based GPUs. In this cloud-computing paradigm, a user must specify their demand for GPUs at every moment in time, and will pay for every GPU-hour they use. ML training jobs are known to be parallelizable to different degrees. Given a stream of ML training jobs, a user typically wants to minimize the mean response time across all jobs. Here, the response time of a job denotes the time from when a job arrives until it is complete. Additionally, the user is constrained by some operating budget. Specifically, in this paper the user is constrained to use no more than $b$ GPUs per hour, over a long-run time average. The question is how to minimize mean response time while meeting the budget constraint. Because training jobs receive a diminishing marginal benefit from running on additional GPUs, allocating too many GPUs to a single training job can dramatically increase the overall cost paid by the user. Hence, an optimal rental policy must balance a tradeoff between training cost and mean response time. This paper derives the optimal rental policy for a stream of training jobs where the jobs have different levels of parallelizability (specified by a speedup function) and different job sizes (amounts of inherent work). We make almost no assumptions about the arrival process and about the job size distribution. Our optimal policy specifies how many GPUs to rent at every moment in time and how to allocate these GPUs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。