arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1705.09073cs.DC
收藏

异构集群上倾斜流的负载均衡

Load Balancing for Skewed Streams on Heterogeneous Cluster

Muhammad Anis Uddin Nasir, Hiroshi Horii, Marco Serafini, Nicolas Kourtellis, Rudy Raymond, Sarunas Girdzijauskas, Takayuki Osogami

摘要:

流应用程序经常遇到不对称的工作负载,并在异构集群上执行。在这种不利条件下的最佳资源利用成为一个挑战,因为它需要推断资源容量和输入分布在运行时间。在本文中,我们通过将其建模为负载均衡问题来解决上述挑战。我们提出了一种新的分区策略,称为一致性分区(CG),它使每个处理元素实例(PEI)处理的工作负载根据其容量。CG背后的主要思想是在源处的小型、大小相等的虚拟工作者的概念,这些虚拟工作者根据其能力分配给工作者。我们提供了一个理论分析所提出的算法,并通过广泛的实证评估表明,我们提出的方案优于国家的最先进的方法,如关键分组。特别是,CG在延迟方面的性能比键分组高3.44倍。

英文摘要:

Streaming applications frequently encounter skewed workloads and execute on heterogeneous clusters. Optimal resource utilization in such adverse conditions becomes a challenge, as it requires inferring the resource capacities and input distribution at run time. In this paper, we tackle the aforementioned challenges by modeling them as a load balancing problem. We propose a novel partitioning strategy called Consistent Grouping (CG), which enables each processing element instance (PEI) to process the workload according to its capacity. The main idea behind CG is the notion of small, equal-sized virtual workers at the sources, which are assigned to workers based on their capacities. We provide a theoretical analysis of the proposed algorithm and show via extensive empirical evaluation that our proposed scheme outperforms the state-of-the-art approaches, like key grouping. In particular, CG achieves 3.44x better performance in terms of latency compared to key grouping.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑