从RDMA到RDCA:使用远程直接缓存访问实现数据中心网络的高速最后一英里
From RDMA to RDCA: Toward High-Speed Last Mile of Data Center Networks Using Remote Direct Cache Access
摘要:
本文通过系统的测量研究表明,现代分布式应用的高内存带宽消耗会导致高速RDMA网络中网络吞吐量的显著下降和尾部延迟的大幅增加,其根本原因是应用进程和网络进程之间的高内存带宽竞争。这种争用导致接收主机的NIC处频繁的数据包丢弃,这触发了网络的拥塞控制机制,并最终导致网络性能下降。为了解决这个问题,我们进行了一个关键的观察,即给定分布式存储服务,它从网络接收的绝大多数数据最终将被写入高速存储介质(例如,SSD)的CPU。因此,我们建议在处理接收到的数据时绕过主机内存,以完全规避此性能瓶颈。特别是,我们设计了Lamda,一种新型的接收器缓存处理系统,消耗少量的CPU缓存来处理从网络中接收到的数据线速率。我们实现了一个原型的Lamda和评估其性能广泛的Clos为基础的测试平台。实验结果表明,在分布式存储应用中,Lamda在存储节点内存带宽消耗为零的情况下,网络吞吐量提高了4.7%,在内存带宽压力下,大数据块和小数据块的网络吞吐量分别提高了17%和45%。Lamda还可以应用于延迟敏感的HPC应用程序,将其通信延迟降低35.1%。
英文摘要:
In this paper, we conduct systematic measurement studies to show that the high memory bandwidth consumption of modern distributed applications can lead to a significant drop of network throughput and a large increase of tail latency in high-speed RDMA networks.We identify its root cause as the high contention of memory bandwidth between application processes and network processes. This contention leads to frequent packet drops at the NIC of receiving hosts, which triggers the congestion control mechanism of the network and eventually results in network performance degradation. To tackle this problem, we make a key observation that given the distributed storage service, the vast majority of data it receives from the network will be eventually written to high-speed storage media (e.g., SSD) by CPU. As such, we propose to bypass host memory when processing received data to completely circumvent this performance bottleneck. In particular, we design Lamda, a novel receiver cache processing system that consumes a small amount of CPU cache to process received data from the network at line rate. We implement a prototype of Lamda and evaluate its performance extensively in a Clos-based testbed. Results show that for distributed storage applications, Lamda improves network throughput by 4.7% with zero memory bandwidth consumption on storage nodes, and improves network throughput by up 17% and 45% for large block size and small size under the memory bandwidth pressure, respectively. Lamda can also be applied to latency-sensitive HPC applications, which reduces their communication latency by 35.1%.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。