arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.14241cs.DCcs.LG
收藏

HPC系统在线故障分类的机器学习方法

A Machine Learning Approach to Online Fault Classification in HPC Systems

Leibniz Supercomputing Centre(莱布尼茨超级计算中心);University of Bologna(博洛尼亚大学);University of Pisa(比萨大学)

Alessio Netti, Zeynep Kiziltan, Ozalp Babaoglu, Alina Sirbu, Andrea Bartolini, Andrea Borghesi

摘要:

随着高性能计算(HPC)系统朝着亿级目标努力,硬件和软件级别的故障率将显著增加。因此,在HPC系统中发生故障时对其进行检测和分类,并在其转变为故障之前启动纠正措施,对于持续运行至关重要。这一目标的核心是故障注入,这是故意触发系统中的故障,以便在受控环境中观察它们的行为。本文提出了一种基于机器学习的高性能计算系统故障分类方法。我们的方法的新颖性在于它可以在线方式对流式数据进行操作,从而打开了在目标系统上实时设计和制定控制动作的可能性。我们介绍了一个高层次的,易于使用的故障注入工具,称为FINJ,重点是复杂的实验管理。为了训练和评估我们的机器学习分类器,我们注入故障的内部实验HPC系统使用FINJ,并生成故障数据集,我们广泛描述。FINJ和数据集都是公开的,以促进HPC系统领域的弹性研究。实验结果表明,我们的方法允许几乎完美的分类精度达到不同的故障类型与低的计算开销和最小的延迟。

英文摘要:

As High-Performance Computing (HPC) systems strive towards the exascale goal, failure rates both at the hardware and software levels will increase significantly. Thus, detecting and classifying faults in HPC systems as they occur and initiating corrective actions before they can transform into failures becomes essential for continued operation. Central to this objective is fault injection, which is the deliberate triggering of faults in a system so as to observe their behavior in a controlled environment. In this paper, we propose a fault classification method for HPC systems based on machine learning. The novelty of our approach rests with the fact that it can be operated on streamed data in an online manner, thus opening the possibility to devise and enact control actions on the target system in real-time. We introduce a high-level, easy-to-use fault injection tool called FINJ, with a focus on the management of complex experiments. In order to train and evaluate our machine learning classifiers, we inject faults to an in-house experimental HPC system using FINJ, and generate a fault dataset which we describe extensively. Both FINJ and the dataset are publicly available to facilitate resiliency research in the HPC systems field. Experimental results demonstrate that our approach allows almost perfect classification accuracy to be reached for different fault types with low computational overhead and minimal delay.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑