arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.17036cs.DC
收藏

Union:一种用于加速网络仿真的自动工作负载管理器

Union: An Automatic Workload Manager for Accelerating Network Simulation

Xin Wang, Misbah Mubarak, Yao Kang, Robert B. Ross, Zhiling Lan

AI总结:

本文提出自动工作负载管理器Union,结合CODES在蜻蜓系统上研究科学仿真与机器学习混合负载的网络干扰,发现HPC应用性能更受消息延迟变化影响,而ML应用更依赖通信时间。

摘要:

随着机器学习应用的快速增长,未来HPC系统的工作负载预计将是科学仿真、大数据分析和机器学习应用的混合体。仿真是一种重要的研究手段,可用于理解在大规模系统上科学应用与大数据及机器学习工作负载共同运行时对性能的影响。本文提出了Union,一个工作负载管理器,它提供了一个自动化框架,以便在CODES中促进混合工作负载仿真。此外,我们利用Union与CODES,在两个蜻蜓(dragonfly)系统上研究了由传统仿真应用和新兴学习应用组成的各种混合工作负载。实验结果表明,消息延迟和通信时间都是评估网络干扰的重要性能指标。网络干扰对HPC应用的影响更多地通过消息延迟变化来体现,而ML应用的性能则更多地取决于通信时间。

英文摘要:

With the rapid growth of the machine learning applications, the workloads of future HPC systems are anticipated to be a mix of scientific simulation, big data analytics, and machine learning applications. Simulation is a great research vehicle to understand the performance implications of co-running scientific applications with big data and machine learning workloads on large-scale systems. In this paper, we present Union, a workload manager that provides an automatic framework to facilitate hybrid workload simulation in CODES. Furthermore, we use Union, along with CODES, to investigate various hybrid workloads composed of traditional simulation applications and emerging learning applications on two dragonfly systems. The experiment results show that both message latency and communication time are important performance metrics to evaluate network interference. Network interference on HPC applications is more reflected by the message latency variation, whereas ML application performance depends more on the communication time.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑