PaPaS:一个可移植的、轻量级的、通用的并行参数研究框架
PaPaS: A Portable, Lightweight, and Generic Framework for Parallel Parameter Studies
摘要:
当前的科学研究领域广泛地基于建模和仿真,通常具有仿真执行流和参数化属性的复杂性。执行流程不一定是简单的,因为它们可能需要多个处理任务和迭代。此外,参数和性能研究是用于表征仿真的常用方法,通常需要遍历大的参数空间。高性能计算机提供了实用的资源,但代价是用户需要处理作业的设置、提交和管理。这项工作提出了设计的PaPaS,一个便携式的,轻量级的,通用的工作流程框架进行并行参数和性能研究。工作流是使用基于关键字-值对语法的参数文件定义的,因此消除了用户创建复杂脚本来管理工作流的开销。参数集由环境变量、文件、部分文件内容和命令行参数的任意组合组成。PaPaS是用Python 3开发的,支持使用SSH、批处理系统和C++ MPI的分布式并行化。PaPaS框架将作为用户进程运行,并可用于单/多节点和多租户计算系统。使用NetLogo的BehaviorSpace工具和使用OpenMP的矩阵乘法的示例模拟分别作为参数和性能研究。结果表明,PaPaS框架提供了一个简单的方法来定义和管理参数的研究,同时提高资源利用率。
英文摘要:
The current landscape of scientific research is widely based on modeling and simulation, typically with complexity in the simulation's flow of execution and parameterization properties. Execution flows are not necessarily straightforward since they may need multiple processing tasks and iterations. Furthermore, parameter and performance studies are common approaches used to characterize a simulation, often requiring traversal of a large parameter space. High-performance computers offer practical resources at the expense of users handling the setup, submission, and management of jobs. This work presents the design of PaPaS, a portable, lightweight, and generic workflow framework for conducting parallel parameter and performance studies. Workflows are defined using parameter files based on keyword-value pairs syntax, thus removing from the user the overhead of creating complex scripts to manage the workflow. A parameter set consists of any combination of environment variables, files, partial file contents, and command line arguments. PaPaS is being developed in Python 3 with support for distributed parallelization using SSH, batch systems, and C++ MPI. The PaPaS framework will run as user processes, and can be used in single/multi-node and multi-tenant computing systems. An example simulation using the BehaviorSpace tool from NetLogo and a matrix multiply using OpenMP are presented as parameter and performance studies, respectively. The results demonstrate that the PaPaS framework offers a simple method for defining and managing parameter studies, while increasing resource utilization.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。