arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.08803cs.DCcs.AI
收藏

云服务支持跨异构资源的高效AI引导模拟工作流

Cloud Services Enable Efficient AI-Guided Simulation Workflows across Heterogeneous Resources

Argonne National Laboratory(阿贡国家实验室);University of Chicago(芝加哥大学);Pacific Northwest National Laboratory(太平洋西北国家实验室)

Logan Ward, J. Gregory Pauloski, Valerie Hayot-Sasson, Ryan Chard, Yadu Babuji, Ganesh Sivaraman, Sutanay Choudhury, Kyle Chard, Rajeev Thakur, Ian Foster

AI总结:

本文提出利用云托管管理服务解决跨异构资源身份验证、FaaS调用与数据传输问题,并通过按引用传数据和可配置引导算法隐藏延迟,实现与直连系统相当的性能,推动异构资源在计算科学中的常规应用。

摘要:

融合机器学习与模拟的应用可以受益于多种计算资源的使用,例如模拟代码运行在高度并行的超级计算机上,而AI训练和推理任务运行在专用加速器上。在此,我们展示了在两个此类异构系统上部署两个AI引导模拟工作流的经验。我们方法的一个独特之处在于使用云托管管理服务来处理跨资源身份验证与授权、函数即服务(FaaS)函数调用以及数据传输等具有挑战性的方面。我们证明这些方法能够达到与依赖资源间直接连接的系统相当的性能。我们通过将FaaS系统和数据传输能力与一个在管理者和工作者之间按引用传递数据的系统集成,并采用用户可配置的引导算法来隐藏数据传输延迟,从而实现性能相当。我们预计这种易用性可以推动异构资源在计算科学中的常规使用。

英文摘要:

Applications that fuse machine learning and simulation can benefit from the use of multiple computing resources, with, for example, simulation codes running on highly parallel supercomputers and AI training and inference tasks on specialized accelerators. Here, we present our experiences deploying two AI-guided simulation workflows across such heterogeneous systems. A unique aspect of our approach is our use of cloud-hosted management services to manage challenging aspects of cross-resource authentication and authorization, function-as-a-service (FaaS) function invocation, and data transfer. We show that these methods can achieve performance parity with systems that rely on direct connection between resources. We achieve parity by integrating the FaaS system and data transfer capabilities with a system that passes data by reference among managers and workers, and a user-configurable steering algorithm to hide data transfer latencies. We anticipate that this ease of use can enable routine use of heterogeneous resources in computational science.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑