arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1507.07429cs.DChep-ex
收藏

通过Apache Mesos优化CMS构建基础架构

Optimizing CMS build infrastructure via Apache Mesos

David Abdurachmanov, Alessandro Degano, Peter Elmer, Giulio Eulisse, David Mendez, Shahzad Muzaffar

摘要:

CERN大型强子对撞机(LHC)CMS实验的离线软件由近1000名物理学家在十年内开发的600万行内部代码组成,以及相当数量的通用开源代码。WLCG的成功构建和早期运营的一个关键因素是在2000年左右融合了商用x86-64处理器和Linux的同构环境。Apache Mesos是一个集群管理器,它在分布式应用程序或框架之间提供高效的资源隔离和共享。它可以在动态共享的节点池上运行Hadoop、Jenkins、Spark、Aurora和其他应用程序。我们介绍了我们如何迁移我们的continuos集成系统,以在一个相对较小的Apache Mesos启用集群上调度作业,以及如何由于Mesos的动态调度功能而导致更好的资源利用率,更高的峰值性能和更低的延迟。

英文摘要:

The Offline Software of the CMS Experiment at the Large Hadron Collider (LHC) at CERN consists of 6M lines of in-house code, developed over a decade by nearly 1000 physicists, as well as a comparable amount of general use open-source code. A critical ingredient to the success of the construction and early operation of the WLCG was the convergence, around the year 2000, on the use of a homogeneous environment of commodity x86-64 processors and Linux. Apache Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications, or frameworks. It can run Hadoop, Jenkins, Spark, Aurora, and other applications on a dynamically shared pool of nodes. We present how we migrated our continuos integration system to schedule jobs on a relatively small Apache Mesos enabled cluster and how this resulted in better resource usage, higher peak performance and lower latency thanks to the dynamic scheduling capabilities of Mesos.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑