利用多人的力量:可扩展的嵌入式应用程序的可扩展工具包
Harnessing the Power of Many: Extensible Toolkit for Scalable Ensemble Applications
摘要:
许多科学问题需要执行多个不同的计算任务,以实现所需的解决方案。我们引入Encoder Toolkit(EnTK)来解决它们带来的规模,多样性和可靠性挑战。我们描述了EnTK的设计和实现,描述了其性能,并将其与两个不同的示例用例相结合:地震反演和自适应模拟集成。我们进行了九个实验,表征EnTK开销,强大和薄弱的可扩展性,以及两个用例实现的性能,在规模和生产基础设施。我们展示了EnTK如何满足以下一般要求:(一)实现专用的抽象来支持集成应用程序的描述和执行;(二)支持异构计算基础设施上的执行;(三)有效的可扩展性高达O(10^4)任务;(四)容错。我们讨论了新的计算能力,EnTK使其产生的科学优势。我们建议EnTK作为一个重要的补充套件的工具,以支持生产科学计算。
英文摘要:
Many scientific problems require multiple distinct computational tasks to be executed in order to achieve a desired solution. We introduce the Ensemble Toolkit (EnTK) to address the challenges of scale, diversity and reliability they pose. We describe the design and implementation of EnTK, characterize its performance and integrate it with two distinct exemplar use cases: seismic inversion and adaptive analog ensembles. We perform nine experiments, characterizing EnTK overheads, strong and weak scalability, and the performance of two use case implementations, at scale and on production infrastructures. We show how EnTK meets the following general requirements: (i) implementing dedicated abstractions to support the description and execution of ensemble applications; (ii) support for execution on heterogeneous computing infrastructures; (iii) efficient scalability up to O(10^4) tasks; and (iv) fault tolerance. We discuss novel computational capabilities that EnTK enables and the scientific advantages arising thereof. We propose EnTK as an important addition to the suite of tools in support of production scientific computing.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。