arXiv 1602.014122016-02cs.DC

Canary：一种面向高性能云计算的调度架构

Canary: A Scheduling Architecture for High Performance Cloud Computing

Hang Qu, Omid Mashayekhi, David Terei, Philip Levis

备注：: We have some presentation issues with the paper

摘要：

我们提出了Canary，一种调度架构，允许高性能分析工作负载扩展到数千个核心上运行。Canary的动机是观察到中央调度器是高性能代码的瓶颈：少数多核工作者执行任务的速度比控制器调度它们的速度更快。Canary的关键见解是颠倒控制器和工作人员之间的责任。在Canary中，控制器将数据分区分配给工作者，然后工作者在本地产生和调度任务，而不是将任务分派给工作者，然后工作者根据需要获取数据。我们在Canary中评估了三个基准测试应用程序，这些应用程序在Amazon EC2上运行多达64台服务器和1，152个核心。Canary的速度比Spark快9- 90倍，比高度优化的图形分析引擎GraphX快4倍。虽然目前的集中式处理器可以每秒调度2，500个任务，但每个Canary工作器可以在每个核心上每秒调度136，000个任务，实验表明这是线性扩展的，64个工作器每秒调度超过1.2亿个任务，允许Canary支持在数千个核心上运行的优化作业。

英文摘要：

We present Canary, a scheduling architecture that allows high performance analytics workloads to scale out to run on thousands of cores. Canary is motivated by the observation that a central scheduler is a bottleneck for high performance codes: a handful of multicore workers can execute tasks faster than a controller can schedule them. The key insight in Canary is to reverse the responsibilities between controllers and workers. Rather than dispatch tasks to workers, which then fetch data as necessary, in Canary the controller assigns data partitions to workers, which then spawn and schedule tasks locally. We evaluate three benchmark applications in Canary on up to 64 servers and 1,152 cores on Amazon EC2. Canary achieves up to 9-90X speedup over Spark and up to 4X speedup over GraphX, a highly optimized graph analytics engine. While current centralized schedulers can schedule 2,500 tasks/second, each Canary worker can schedule 136,000 tasks/second per core and experiments show this scales out linearly, with 64 workers scheduling over 120 million tasks per second, allowing Canary to support optimized jobs running on thousands of cores.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文