UMDAM: 一种统一的数据布局和DRAM地址映射方案用于异构NPU-PIM
UMDAM: A Unified Data Layout and DRAM Address Mapping for Heterogenous NPU-PIM
AI总结:
UMDAM通过统一的数据布局和DRAM映射方案,解决NPU-PIM协同计算中的数据布局不匹配、带宽损失和冗余存储问题,提升边缘设备LLM推理效率。
摘要:
大型语言模型(LLMs)越来越多地部署在配备神经处理单元(NPUs)的边缘设备上,但解码阶段仍存在内存密集型问题,限制了性能。处理在内存中(PIM)提供了一个有前景的解决方案,但协同执行NPU-PIM系统面临数据布局不匹配、带宽损失和冗余存储等挑战。为此,我们提出UMDAM,一种针对NPU-PIM协同执行的统一内存亲和数据布局和DRAM地址映射方案。UMDAM采用列优先、分块布局和可配置的DRAM映射策略,确保与NPU计算的兼容性,同时最大化PIM效率——无需引入额外的内存开销或带宽损失。在OPT模型上的综合评估表明,UMDAM将时间到第一个token(TTFT)减少了高达3.0倍,时间到最后一个token(TTLT)减少了2.18倍,显著提高了边缘设备上的LLM推理效率。
英文摘要:
Large Language Models (LLMs) are increasingly deployed on edge devices with Neural Processing Units (NPUs), yet the decode phase remains memory-intensive, limiting performance. Processing-in-Memory (PIM) offers a promising solution, but co-executing NPU-PIM systems face challenges such as data layout mismatches, bandwidth loss, and redundant storage. To address these issues, we propose UMDAM, a unified memory-affinity data layout and DRAM address mapping scheme tailored for NPU-PIM co-execution. UMDAM employs a column-major, tile-based layout and a configurable DRAM mapping strategy to ensure compatibility with NPU computation while maximizing PIM efficiency -- without introducing extra memory overhead or bandwidth loss. Comprehensive evaluations on OPT models demonstrate that UMDAM reduces time-to-first-token (TTFT) by up to 3.0x and time-to-last-token (TTLT) by 2.18x, significantly improving end-to-end LLM inference efficiency on edge devices.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。