arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.03451cs.DCcs.LGcs.PF
收藏

用于性能预测的纵向系统监控数据分析

Analytics of Longitudinal System Monitoring Data for Performance Prediction

Google, Inc.(谷歌公司);University of Maryland(马里兰大学)

Ian J. Costello, Abhinav Bhatele

AI总结:

本文利用 HPC 全系统纵向监控数据与机器学习建模控制作业性能,识别主导预测特征,并证明模型可应用无关地预测未训练应用的性能。

摘要:

近年来,多个 HPC(高性能计算)设施已开始持续监控其系统和作业,以收集与性能相关的数据,从而理解性能和运行效率。通过构建数据驱动模型来预测调度队列中等待作业的性能,此类数据可用于优化单个作业及整个系统的性能。本文中,我们利用纵向的全系统监控数据和机器学习对代表性控制作业的性能进行建模,以探究性能可变性的成因。我们非常详细地分析了这些预测模型,以识别作为性能主导预测因子的特征。我们证明,此类模型可以与应用无关,并可用于预测未包含在训练中的应用的性能。

英文摘要:

In recent years, several HPC facilities have started continuous monitoring of their systems and jobs to collect performance-related data for understanding performance and operational efficiency. Such data can be used to optimize the performance of individual jobs and the overall system by creating data-driven models that can predict the performance of jobs waiting in the scheduler queue. In this paper, we model the performance of representative control jobs using longitudinal system-wide monitoring data and machine learning to explore the causes of performance variability. We analyze these prediction models in great detail to identify the features that are dominant predictors of performance. We demonstrate that such models can be application-agnostic and can be used for predicting performance of applications that are not included in training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑