引入JIRIAF:一种用于优化HPC资源供应的虚拟Kubelet集成
Introducing JIRIAF: A Virtual Kubelet Integration for Optimizing HPC Resource Provisioning
AI总结:
本文介绍JIRIAF框架及其资源管理器JRM,利用Kubernetes和Virtual Kubelet动态管理HPC资源,并在Perlmutter系统上验证其高效性,同时集成数字孪生与DBN以优化监控。
摘要:
JIRIAF(JLab跨设施综合研究基础设施)框架旨在简化资源管理,并优化异构环境中的高性能计算(HPC)工作负载。JIRIAF的核心是JIRIAF资源管理器(JRM),它有效利用Kubernetes和Virtual Kubelet动态管理资源,即使在用户权限受限的环境中也能如此。通过在用户空间运行,JRM促进了用户应用程序作为容器在不同计算站点上的执行,确保了统一的控制和监控。该框架的有效性通过一个案例研究得到证明,该案例涉及在NERSC的Perlmutter系统上部署数据流处理管道,展示了其高效管理大规模HPC应用的能力。此外,我们讨论了与流系统相关的模拟队列系统的数字孪生模型的集成,使用动态贝叶斯网络(DBN)来增强实时监控和控制,为系统性能和优化策略提供了宝贵的见解。
英文摘要:
The JIRIAF (JLab Integrated Research Infrastructure Across Facilities) framework is designed to streamline resource management and optimize high-performance computing (HPC) workloads across heterogeneous environments. Central to JIRIAF is the JIRIAF Resource Manager (JRM), which effectively leverages Kubernetes and Virtual Kubelet to manage resources dynamically, even in environments with restricted user privileges. By operating in userspace, JRM facilitates the execution of user applications as containers across diverse computing sites, ensuring unified control and monitoring. The framework's effectiveness is demonstrated through a case study involving the deployment of data-stream processing pipelines on the Perlmutter system at NERSC, showcasing its capability to manage large-scale HPC applications efficiently. Additionally, we discuss the integration of a digital twin model for a simulated queue system related to a streaming system, using a Dynamic Bayesian Network (DBN) to enhance real-time monitoring and control, providing valuable insights into system performance and optimization strategies.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。