面向分布内存科学计算的中间语言并行脚本
Toward Interlanguage Parallel Scripting for Distributed-Memory Scientific Computing
摘要:
Python和R等脚本语言由于其强大的功能和表现力以及可用的库,已被广泛用作科学软件的生产性开发工具。然而,在诸如IBM Blue Gene/Q或Cray XE 6的大规模并行计算机系统上部署脚本化应用程序是一个挑战,因为存在包括操作系统限制、互操作性挑战、由于脚本化方法中常见的小文件系统访问而导致的并行文件系统开销以及其他问题的问题。我们在这里提出了一种新的方法来解决这些问题,其中Swift脚本系统用于集成用Python,R和Tcl编写的高级脚本,与用C,C++和Fortran开发的本地代码,通过将Swift链接到脚本解释器的库接口。在这种方法中,Swift处理分布式内存进程之间的数据管理、移动和编组,而无需用户直接操作低级通信库(如MPI)。我们提出了一种技术,有效地启动脚本的应用程序在大型超级计算机上使用分层编程模型。
英文摘要:
Scripting languages such as Python and R have been widely adopted as tools for the productive development of scientific software because of the power and expressiveness of the languages and available libraries. However, deploying scripted applications on large-scale parallel computer systems such as the IBM Blue Gene/Q or Cray XE6 is a challenge because of issues including operating system limitations, interoperability challenges, parallel filesystem overheads due to the small file system accesses common in scripted approaches, and other issues. We present here a new approach to these problems in which the Swift scripting system is used to integrate high-level scripts written in Python, R, and Tcl, with native code developed in C, C++, and Fortran, by linking Swift to the library interfaces to the script interpreters. In this approach, Swift handles data management, movement, and marshaling among distributed-memory processes without direct user manipulation of low-level communication libraries such as MPI. We present a technique to efficiently launch scripted applications on large-scale supercomputers using a hierarchical programming model.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。