arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.11044cs.AR
收藏

异步内存访问单元:利用大规模并行性实现远内存访问

Asynchronous Memory Access Unit: Exploiting Massive Parallelism for Far Memory Access

Luming Wang, Xu Zhang, Songyue Wang, Zhuolun Jiang, Tianyue Lu, Mingyu Chen, Siwei Luo, Keji Huang

AI总结:

针对远内存访问延迟高导致的性能问题,提出异步内存访问指令与配套单元AMU,通过分离请求与响应、复用L2缓存作SPM及协程框架提升MLP,实验验证其可有效掩盖远内存延迟并实现显著加速。

摘要:

现代应用不断增长的内存需求推动了数据中心采用远内存技术,以提供经济高效的高容量内存解决方案。然而,远内存带来了新的性能挑战,因为其访问延迟比本地DRAM长得多,且延迟波动更大。为使应用在远内存上达到可接受的性能,需要高度的内存级并行性(MLP)来容忍长访问延迟。虽然现代乱序处理器能够利用一定程度的MLP,但它们受限于资源约束和硬件复杂度。核心障碍是传统加载/存储指令的同步内存访问语义,这类指令会长时间占用关键硬件资源,而更长的远内存延迟进一步加剧了这一限制。\n本文提出了一套异步内存访问指令(Asynchronous Memory Access Instructions, AMI)及其配套功能单元——异步内存访问单元(Asynchronous Memory Access Unit, AMU),集成于当代乱序核心内部。AMI将内存请求发起与响应处理分离,以减少资源占用。此外,AMU架构通过将部分L2缓存改作便笺式存储器(scratchpad memory, SPM)提供充足的临时存储,可支持多达数百个异步内存请求。结合基于协程的编程框架,该方案能够实现显著更高的MLP,以掩盖远内存延迟。\n采用周期精确仿真的评估结果显示,在远内存额外延迟为1微秒的情况下,AMI对内存受限基准测试实现了平均2.42倍的加速比。在延迟为5微秒的GUPS(随机访问)测试中,该方案支持超过130个未完成请求,实现了26.86倍的加速比。这些结果证明了该技术如何通过显式MLP表达和延迟适配来应对远内存的性能影响。

英文摘要:

The growing memory demands of modern applications have driven the adoption of far memory technologies in data centers to provide cost-effective, high-capacity memory solutions. However, far memory presents new performance challenges because its access latencies are significantly longer and more variable than local DRAM. For applications to achieve acceptable performance on far memory, a high degree of memory-level parallelism (MLP) is needed to tolerate the long access latency. While modern out-of-order processors are capable of exploiting a certain degree of MLP, they are constrained by resource limitations and hardware complexity. The key obstacle is the synchronous memory access semantics of traditional load/store instructions, which occupy critical hardware resources for a long time. The longer far memory latencies exacerbate this limitation. This paper proposes a set of Asynchronous Memory Access Instructions (AMI) and its supporting function unit, Asynchronous Memory Access Unit (AMU), inside a contemporary Out-of-Order Core. AMI separates memory request issuing from response handling to reduce resource occupation. Additionally, AMU architecture supports up to several hundreds of asynchronous memory requests through re-purposing a portion of L2 Cache as scratchpad memory (SPM) to provide sufficient temporal storage. Together with a coroutine-based programming framework, this scheme can achieve significantly higher MLP for hiding far memory latencies. Evaluation with a cycle-accurate simulation shows AMI achieves 2.42x speedup on average for memory-bound benchmarks with 1us additional far memory latency. Over 130 outstanding requests are supported with 26.86x speedup for GUPS (random access) with 5 us latency. These demonstrate how the techniques tackle far memory performance impacts through explicit MLP expression and latency adaptation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑