MLCC：一种加速机器学习训练的拥塞控制技术
MLCC: A Congestion Control Technique to Accelerate ML Training
AI总结：
MLCC是一种分布式拥塞控制技术，通过调整DNN训练流的发送速率实现通信与计算交错，在GPU集群中加速DNN训练，经测试和模拟可显著提升训练迭代速度与吞吐量。
摘要：
我们提出MLCC，这是一种新颖的技术，以完全分布式的方式增强当今的拥塞控制算法，从而加速共享GPU集群中的DNN训练作业。MLCC的核心是一个简单的原则：DNN训练流应调整其发送速率，将其他流的通信转移到自身的计算周期中，实现交错。我们表明，将此原则集成到当今的拥塞控制协议中很简单（对于给定协议需要不到60行代码），并使DNN作业在几次训练迭代内实现交错，从而减少网络争用并缩短作业完成时间。我们的测试台表明，MLCC可将平均和第99百分位训练迭代时间分别加速高达1.9倍和2.7倍。通过广泛的数据包级模拟，我们在36节点、288 GPU的胖树拓扑上观察到训练吞吐量提高了1.35倍。
英文摘要：
We present MLCC, a novel technique to augment today's congestion control algorithms to accelerate DNN training jobs in shared GPU clusters in a fully distributed manner. At the heart of MLCC lies a straightforward principle: DNN training flows should scale their sending rate to shift other flows' communication into their compute periods, achieving interleaving. We show that integrating this principle into today's congestion control protocols is simple (requiring less than 60 lines of code for a given protocol) and enables DNN jobs to interleave within a few training iterations, thereby reducing network contention and improving job completion times. Our testbed demonstrates that MLCC accelerates the average and 99th percentile training iteration times by up to 1.9x and 2.7x respectively. Through extensive packet-level simulations, we observe a 1.35x improvement in training throughput on a 36-node, 288 GPU fat-tree topology.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
