一种数据驱动的计算集群资源分配动态调整方法
A Data-Driven Approach to Dynamically Adjust Resource Allocation for Compute Clusters
摘要:
如今,数据中心在很大程度上未得到充分利用,因为资源分配是基于预留机制的,而预留机制忽略了实际的资源利用率。实际上,为高峰需求预留资源是很常见的,高峰需求可能只在应用程序生命周期的一小部分时间内发生。因此,集群资源经常得不到充分利用。在这项工作中,我们提出了一种机制,提高集群的利用率,从而降低了平均周转时间,同时防止应用程序失败,由于在访问有限的资源,如RAM的竞争。我们的方法监控资源利用率,并采用数据驱动的方法进行资源需求预测,并对预测中的不确定性进行量化。使用需求预测和它的信心,我们的机制调制集群资源分配给运行中的应用程序,并减少了周转时间一个数量级以上,同时保持应用程序故障得到控制。因此,租户享受响应系统,提供商受益于高效的集群利用率。
英文摘要:
Nowadays, data-centers are largely under-utilized because resource allocation is based on reservation mechanisms which ignore actual resource utilization. Indeed, it is common to reserve resources for peak demand, which may occur only for a small portion of the application life time. As a consequence, cluster resources often go under-utilized. In this work, we propose a mechanism that improves cluster utilization, thus decreasing the average turnaround time, while preventing application failures due to contention in accessing finite resources such as RAM. Our approach monitors resource utilization and employs a data-driven approach to resource demand forecasting, featuring quantification of uncertainty in the predictions. Using demand forecast and its confidence, our mechanism modulates cluster resources assigned to running applications, and reduces the turnaround time by more than one order of magnitude while keeping application failures under control. Thus, tenants enjoy a responsive system and providers benefit from an efficient cluster utilization.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。