HammingMesh：一种用于大规模深度学习的网络拓扑
HammingMesh: A Network Topology for Large-Scale Deep Learning
摘要：
大量的微架构优化为深度神经网络释放了巨大的处理能力，从而推动了人工智能革命。随着这种优化的耗尽，现代人工智能的增长现在取决于训练系统的性能，特别是它们的数据移动。我们不是专注于单个加速器，而是在全系统范围内研究大规模训练的数据移动特征。基于我们的工作负载分析，我们设计了HammingMesh，一种新的网络拓扑结构，提供高带宽，低成本，高作业调度的灵活性。具体来说，HammingMesh可以支持全带宽和隔离到具有两个并行维度的深度学习训练作业。此外，它还支持通用流量的高全局带宽。因此，HammingMesh将为未来具有极端带宽要求的大规模深度学习系统提供动力。
英文摘要：
Numerous microarchitectural optimizations unlocked tremendous processing power for deep neural networks that in turn fueled the AI revolution. With the exhaustion of such optimizations, the growth of modern AI is now gated by the performance of training systems, especially their data movement. Instead of focusing on single accelerators, we investigate data-movement characteristics of large-scale training at full system scale. Based on our workload analysis, we design HammingMesh, a novel network topology that provides high bandwidth at low cost with high job scheduling flexibility. Specifically, HammingMesh can support full bandwidth and isolation to deep learning training jobs with two dimensions of parallelism. Furthermore, it also supports high global bandwidth for generic traffic. Thus, HammingMesh will power future large-scale deep learning systems with extreme bandwidth requirements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
