DFabric: 通过CXL-Ethernet混合互连技术实现数据并行应用的扩展
DFabric: Scaling Out Data Parallel Applications with CXL-Ethernet Hybrid Interconnects
AI总结:
DFabric通过CXL-Ethernet混合互连技术实现数据并行应用的扩展,解决跨机架通信带宽限制问题。
摘要:
新兴的互连技术,如CXL和NVLink,已被集成到主机内部拓扑中,以扩展更多加速器并促进它们之间的高效通信,例如GPU。为了与加速器日益增长的计算吞吐量保持同步,互连的链路带宽得到了显著增强,例如CXL 3.0链路的256GBps,这比以太网和InfiniBand网络链路高出一个数量级或更多。因此,当数据密集型任务,如大语言模型训练,在多个主机之间扩展超过互连的范围限制时,性能会受到网络基础设施限制带宽的显著阻碍。我们通过提出DFabric,一种两级互连架构来解决这个问题。首先,DFabric通过互连织网(即CXL织网)将机架的计算单元解耦,以在机架级扩展,从而使它们能够享受机架内部的高效互连。其次,DFabric将主机的网络接口卡(NIC)解耦,并将其集中到具有CXL织网的NIC池中。通过提供与互连带宽相媲美的充足聚合容量,NIC池在机架或超过互连织网范围限制时实现了高效的跨机架通信。然而,当使每个主机能够高效利用NIC池时,本地内存访问会成为瓶颈。为此,DFabric通过解耦主机本地内存并添加更多内存设备来构建一个具有足够带宽的记忆池。我们实现了一个DFabric原型,可以透明地运行应用程序。我们通过运行各种微基准测试和计算密集型应用程序,如DNN和图,验证了其性能提升。
英文摘要:
Emerging interconnects, such as CXL and NVLink, have been integrated into the intra-host topology to scale more accelerators and facilitate efficient communication between them, such as GPUs. To keep pace with the accelerator's growing computing throughput, the interconnect has seen substantial enhancement in link bandwidth, e.g., 256GBps for CXL 3.0 links, which surpasses Ethernet and InfiniBand network links by an order of magnitude or more. Consequently, when data-intensive jobs, such as LLM training, scale across multiple hosts beyond the reach limit of the interconnect, the performance is significantly hindered by the limiting bandwidth of the network infrastructure. We address the problem by proposing DFabric, a two-tier interconnect architecture. We address the problem by proposing DFabric, a two-tier interconnect architecture. First, DFabric disaggregates rack's computing units with an interconnect fabric, i.e., CXL fabric, which scales at rack-level, so that they can enjoy intra-rack efficient interconnecting. Second, DFabric disaggregates NICs from hosts, and consolidates them to form a NIC pool with CXL fabric. By providing sufficient aggregated capacity comparable to interconnect bandwidth, the NIC pool bridges efficient communication across racks or beyond the reach limit of interconnect fabric. However, the local memory accessing becomes the bottleneck when enabling each host to utilize the NIC pool efficiently. To the end, DFabric builds a memory pool with sufficient bandwidth by disaggregating host local memory and adding more memory devices. We have implemented a prototype of DFabric that can run applications transparently. We validated its performance gain by running various microbenchmarks and compute-intensive applications such as DNN and graph.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。