使用Apache Mesos进行资源优化的两阶段集群
Two stage cluster for resource optimization with Apache Mesos
摘要：
由于作业的资源估计是困难的，用户经常高估他们的需求。商业云和学术园区集群都存在资源利用率低和等待时间长的问题，因为用户提供的作业资源估计不准确。我们提出了一种方法来统计估计的实际资源需求的作业在一个小集群运行之前，在一个大集群。对小集群的初始估计为我们提供了一个作业需要多少实际资源的视图。这个初始估计允许我们准确地为队列中的挂起作业分配资源，从而提高吞吐量和资源利用率。在我们的实验中，我们确定了资源利用率估计，内存的平均准确度为90%，CPU的平均准确度为94%，而与Apache Aurora和Apache Mesos上的默认作业提交方法相比，我们的内存利用率平均提高了22%，CPU利用率提高了53%。
英文摘要：
As resource estimation for jobs is difficult, users often overestimate their requirements. Both commercial clouds and academic campus clusters suffer from low resource utilization and long wait times as the resource estimates for jobs, provided by users, is inaccurate. We present an approach to statistically estimate the actual resource requirement of a job in a Little cluster before the run in a Big cluster. The initial estimation on the little cluster gives us a view of how much actual resources a job requires. This initial estimate allows us to accurately allocate resources for the pending jobs in the queue and thereby improve throughput and resource utilization. In our experiments, we determined resource utilization estimates with an average accuracy of 90% for memory and 94% for CPU, while we make better utilization of memory by an average of 22% and CPU by 53%, compared to the default job submission methods on Apache Aurora and Apache Mesos.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
