arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.05047cs.ARcs.AIcs.LG
收藏

大语言模型推理硬件的挑战与研究方向

Challenges and Research Directions for Large Language Model Inference Hardware

Google(谷歌)

Xiaoyu Ma, David Patterson

AI总结:

本文探讨了大语言模型推理硬件面临的挑战,提出了高带宽闪存、内存-逻辑堆叠和低延迟互连等研究方向,以提升内存带宽和通信效率。

摘要:

大语言模型(LLM)推理具有挑战性。底层Transformer模型的自回归解码阶段使LLM推理本质上不同于训练。近年来人工智能趋势的加剧,使得主要挑战是内存和互连,而非计算。为解决这些挑战,我们强调四个架构研究机会:具有10倍内存容量的高带宽闪存,具有HBM-like带宽的内存;靠近处理的内存和3D内存-逻辑堆叠,以提高内存带宽;以及低延迟互连以加速通信。尽管我们的关注点是数据中心人工智能,我们也回顾了它们在移动设备上的适用性。

英文摘要:

Large Language Model (LLM) inference is hard. The autoregressive Decode phase of the underlying Transformer model makes LLM inference fundamentally different from training. Exacerbated by recent AI trends, the primary challenges are memory and interconnect rather than compute. To address these challenges, we highlight four architecture research opportunities: High Bandwidth Flash for 10X memory capacity with HBM-like bandwidth; Processing-Near-Memory and 3D memory-logic stacking for high memory bandwidth; and low-latency interconnect to speedup communication. While our focus is datacenter AI, we also review their applicability for mobile devices.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑