JobPruner：一个用于在HPC应用中探索参数空间的机器学习助手
JobPruner: A Machine Learning Assistant for Exploring Parameter Spaces in HPC Applications
摘要：
高性能计算（HPC）应用程序对于科学家和工程师创建和理解模型及其属性至关重要。这些专业人员依赖于执行大型计算作业集，这些作业集探索参数值的组合。避免执行不必要的作业不仅可以加快这些实验的速度，还可以减少基础设施的使用，这一点尤其重要，因为这些应用程序已转移到HPC云平台。我们的假设是，这些实验产生的数据可以帮助用户识别这些工作。为了解决这一假设，我们需要了解工作淘汰决策所需的多个实验之间的相似性水平，以及自动化这一过程所需的步骤。在本文中，我们提出了一项研究和一个基于机器学习的工具，称为JobPruner，以支持参数探索HPC实验。该工具通过来自不同领域的三个真实用例进行了评估，包括地震分析和农学。我们观察到该工具在一次实验中减少了93%的工作，同时在大多数情况下提高了质量。此外，即使考虑到过去相关性较低的实验，也有可能减少工作执行。
英文摘要：
High Performance Computing (HPC) applications are essential for scientists and engineers to create and understand models and their properties. These professionals depend on the execution of large sets of computational jobs that explore combinations of parameter values. Avoiding the execution of unnecessary jobs brings not only speed to these experiments, but also reductions in infrastructure usage---particularly important due to the shift of these applications to HPC cloud platforms. Our hypothesis is that data generated by these experiments can help users in identifying such jobs. To address this hypothesis we need to understand the similarity levels among multiple experiments necessary for job elimination decisions and the steps required to automate this process. In this paper we present a study and a machine learning-based tool called JobPruner to support parameter exploration in HPC experiments. The tool was evaluated with three real-world use cases from different domains including seismic analysis and agronomy. We observed the tool reduced 93% of jobs in a single experiment, while improving quality in most scenarios. In addition, reduction in job executions was possible even considering past experiments with low correlations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
