分布式训练任务网络争用建模的精确模拟
Accurate Simulation of Distributed Training Jobs with Network Contention Modeling
浏览论文内容
中文总结 AI 辅助
针对现有分布式训练模拟器忽略网络争用导致高误差的问题，提出MoSim模拟器，结合无GPU特征化与网络争用模型，显著降低JCT和完工时间模拟误差，并减少输入构建开销。
中文摘要 AI 辅助
基于轨迹的模拟被广泛用于评估GPU集群中的分布式训练（DT）任务，但现有模拟器要么忽略网络争用，要么用固定惩罚近似网络争用。这忽略了调度决策如何决定哪些任务共享服务器网络接口和服务器间链路，从而改变训练期间的网络时间。因此，我们的动机实验表明，它们会产生较大误差，在平均任务完成时间（JCT）上达到高达73.64%的平均绝对百分比误差（MAPE）。本文介绍了MoSim，一种在动态网络争用下模拟DT任务执行的GPU集群模拟器。MoSim将无GPU特征化与网络争用模型相结合：它无需GPU即可获取每个任务的计算时间、网络时间和网络数据量，然后利用当前工作节点分配来估计共享网络接口如何影响每个任务的迭代时间。我们的评估表明，与现有模拟器相比，MoSim将平均JCT的模拟误差降低了最多3.28倍，尾部（第99百分位）JCT降低了最多7.79倍，完工时间降低了最多8.48倍，同时以平均仅8.63%的误差对NIC争用因子进行建模。通过避免真实GPU剖析，MoSim还将输入构建开销降低了44.6倍。
英文摘要
Trace-driven simulation is widely used to evaluate distributed training (DT) jobs in GPU clusters, but existing simulators either ignore network contention or approximate it with a fixed penalty. This misses how scheduling decisions determine which jobs share server network interfaces and inter-server links, thereby changing networking time during training. As a result, our motivating experiments demonstrate that they incur large errors, reaching up to 73.64% mean absolute percentage error (MAPE) in average job completion time (JCT). This paper introduces MoSim, a GPU-cluster simulator that models DT job execution under dynamic network contention. MoSim combines GPU-free characterization with network contention model: it obtains each job's compute time, networking time, and networking volume without GPUs, then uses the current worker assignment to estimate how shared network interfaces affect each job's iteration time. Our evaluation shows that, compared with existing simulators, MoSim reduces simulation error for average JCT by up to 3.28$\times$, tail (99th-percentile) JCT by up to 7.79$\times$, and makespan by up to 8.48$\times$, while modeling NIC contention factors with only 8.63% error on average. By avoiding real-GPU profiling, MoSim also reduces input construction overhead by 44.6$\times$.
发表机构
- Dongguk University（东国大学）
- Korea University（高丽大学）
机构由 AI 辅助整理，请以论文原文为准。
