通过HammerCloud和Big PanDA使用现代HEP基准套件优化ATLAS计算资源使用
Optimisation of ATLAS computing resource usage through a modern HEP Benchmark Suite via HammerCloud and Big PanDA
AI总结:
本文利用HammerCloud和PanDA在136个站点部署增强型HEP Benchmark Suite,系统评估HEPScore23基准下ATLAS计算资源的实际性能与声明能力差异,识别配置问题并优化资源使用效率。
摘要:
2023年4月,基于HEP特定应用的新基准HEPScore23被WLCG采纳,取代了HEP-SPEC06。作为向新基准过渡的一部分,需要将各站点公布的CPU核心算力与运行ATLAS工作负载时观测到的有效算力进行比较。其中一个目标是验证旧基准与新基准分数之间的转换率。另一个目标是理解HEPScore在多核作业槽上运行时的表现,即完全按照计算站点在生产环境中的使用方式进行评估。我们的研究利用HammerCloud基础设施和PanDA工作负载管理系统,使用增强型HEP Benchmark Suite在136个计算站点上收集大规模基准统计数据。该套件不仅使我们能够收集性能指标,而且借助插件,还可以在执行过程中收集机器负载、内存使用情况以及其他用户自定义指标等信息,并将其存储在OpenSearch数据库中。这些广泛的测试使得我们能够对站点的实际计算能力与声明计算能力进行深入分析。研究结果为承诺给ATLAS的计算资源的实际性能提供了有价值的见解,识别出需要改进的领域,同时突出表现不佳或超出预期的站点。此外,这有助于确保各站点的高效运维实践。所收集的指标使我们能够检测并修复配置问题,从而提升实际体验性能。
英文摘要:
In April 2023, HEPScore23, the new benchmark based on HEP specific applications, was adopted by WLCG, replacing HEP-SPEC06. As part of the transition to the new benchmark, the CPU corepower published by the sites needed to be compared with the effective power observed while running ATLAS workloads. One aim was to verify the conversion rate between the scores of the old and the new benchmark. The other objective was to understand how the HEPScore performs when run on multi-core job slots, so exactly like the computing sites are being used in the production environment. Our study leverages the HammerCloud infrastructure and the PanDA Workload Management System to collect a large benchmark statistic across 136 computing sites using an enhanced HEP Benchmark Suite. It allows us to collect not only performance metrics, but, thanks to plugins, it also collects information such as machine load, memory usage and other user-defined metrics during the execution and stores it in an OpenSearch database. These extensive tests allow for an in-depth analysis of the actual, versus declared computing capabilities of these sites. The results provide valuable insights into the real-world performance of computing resources pledged to ATLAS, identifying areas for improvement while spotlighting sites that underperform or exceed expectations. Moreover, this helps to ensure efficient operational practices across sites. The collected metrics allowed us to detect and fix configuration issues and therefore improve the experienced performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。