arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2212.06352cs.DC
收藏

高性能计算中AI模型的无缝管理

Towards Seamless Management of AI Models in High-Performance Computing

Sixing Yu, Murali Emani, Chunhua Liao, Pei-Hung Lin, Tristan Vanderbruggen, Xipeng Shen, Ali Jannesari

摘要:

随着人工智能(AI)在不同科学/工程社区中的日益普及,AI模型在各个领域中以前所未有的规模出现。然而,考虑到软件和硬件环境的复杂性和多样性,重用人工智能工件(模型和数据集)极具挑战性,特别是在人工智能驱动的科学应用中。构建一个生态系统来大规模有效地运行和重用AI应用程序/数据集,对于各种科学和工程以及高性能计算(HPC)社区来说变得越来越重要。在本文中,我们在HPC-AI生态系统- HPCFair上进行了创新,该生态系统实现了Findable,Interoperable,Reproducible(FAIR)原则。HPCFair支持AI模型/数据集的收集,允许用户下载/上传带有身份验证的AI工件。最重要的是,我们提出的框架为用户提供了用户友好的API,让用户可以轻松地运行推理作业,并根据需要为他们的任务定制AI工件。我们的研究结果表明,使用HPCFair API,用户无论在AI方面的技术专业知识如何,都可以轻松地将AI工件应用于他们的任务。

英文摘要:

With the increasing prevalence of artificial intelligence (AI) in diverse science/engineering communities, AI models emerge on an unprecedented scale among various domains. However, given the complexity and diversity of the software and hardware environments, reusing AI artifacts (models and datasets) is extremely challenging, especially with AI-driven science applications. Building an ecosystem to run and reuse AI applications/datasets at scale efficiently becomes increasingly essential for diverse science and engineering and high-performance computing (HPC) communities. In this paper, we innovate over an HPC-AI ecosystem -- HPCFair, which enables the Findable, Accessible, Interoperable, and Reproducible (FAIR) principles. HPCFair enables the collection of AI models/datasets allowing users to download/upload AI artifacts with authentications. Most importantly, our proposed framework provides user-friendly APIs for users to easily run inference jobs and customize AI artifacts to their tasks as needed. Our results show that, with HPCFair API, users irrespective of technical expertise in AI, can easily leverage AI artifacts to their tasks with minimal effort.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑