NITRO：基于3D NAND闪存的高性能存内计算架构，具备增强型激活数据流
NITRO: High-Performance 3D NAND Flash-Based In-Storage Computing with Enhanced Activation Dataflow
浏览论文内容
中文总结 AI 辅助
针对NAND-PIM方案中间值数据流/缓冲的局限，提出带增强型激活缓冲的高性能NAND闪存ISC架构，结合分布式数据流方法，使推理延迟最高降低85%。
中文摘要 AI 辅助
存内计算（ISC）被视为下一代内存架构，因其可缓解主机与内存之间的数据瓶颈。近年来，大语言模型（LLMs）所需资源大幅增加，但内存密度并未同步扩展。近期多项研究探索了基于NAND闪存的内存处理（NAND-PIM）方案，以利用该内存的高密度特性，但这些方案未解决中间值的数据流/缓冲问题，简单方法是在低速闪存阵列中处理这些值。为克服此局限，我们提出一种基于NAND闪存的高性能ISC架构，具备增强型激活缓冲能力。该架构不使用极慢的闪存阵列存储中间值，而是将其缓冲在快速DRAM子系统中，有效解决了激活值写入较慢的TLC NAND闪存时的高延迟惩罚问题。我们还为NAND-PIM阵列引入分布式数据流方法，通过高效的平面内数据映射最大化计算并行度。实验结果显示，与基线相比，所提架构实现了显著的性能提升，推理延迟最高降低85%。
英文摘要
In-storage computing (ISC) is considered a next-generation memory architecture for its ability to relieve the data bottleneck between the host and the memory. While the required resources of large language models (LLMs) have increased significantly in recent years, the memory density has not scaled accordingly. Recently, several works have studied NAND flash-based processing-in-memory (NAND-PIM) schemes to exploit the high density of the memory. However, they do not address the dataflow/buffer for the intermediate values, so a simple method is to deal with the values in the slow flash memory array. To overcome such a limitation, we propose a high-performance NAND flash-based ISC architecture with enhanced activation buffering. Instead of using the very slow flash memory array for the intermediate values, our architecture buffers the values in a fast DRAM subsystem. This approach effectively handles the high-latency penalties when activations are programmed into slower TLC NAND flash. We also introduce a distributed dataflow approach for the NAND-PIM array. This approach maximizes computational parallelism by employing efficient intra-plane data mapping. The results show that our proposed architecture achieves significant performance improvements, reducing the inference latency by up to 85% compared to the baseline.
