arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2511.06174cs.ARcs.AI
收藏

LUT-LLM: 通过FPGA上的基于内存的计算实现高效的大型语言模型推断

LUT-LLM: Efficient Large Language Model Inference with Memory-based Computations on FPGAs

University of California, Los Angeles(加州大学洛杉矶分校)

Zifan He, Shengyu Ye, Rui Ma, Yang Wang, Jason Cong

AI总结:

本文提出LUT-LLM,首个利用基于内存计算的FPGA加速器,通过向量量化部署10亿参数语言模型,提升FPGA效率以竞争GPU性能。

摘要:

大型语言模型(LLM)的快速发展显著提升了日常应用。尽管许多基于FPGA的加速器在细粒度数据控制方面具有灵活性,其速度和能效优于GPU,但最近的GPU专用优化削弱了这一优势。当局限于算术计算时,FPGA往往由于计算资源相对较少而落后于GPU。为解决这一挑战,我们利用FPGA相对于GPU的优势:嵌入在计算单元中的丰富分布式芯片内存。我们相信通过将LLM推断从基于算术的计算转换为基于内存的计算,通过查找表可以提高FPGA的效率以与GPU竞争。然而,现有方法由于算法和架构设计的限制,效率低下或无法扩展和部署语言模型。本文介绍了LUT-LLM,首个部署10亿参数语言模型的FPGA加速器,利用向量量化。我们构建了性能模型,评估了多种量化方案,并确定激活-权重向量共量化为最有效的方法。为支持此方案,LUT-LLM具有(1)带宽感知的并行质心搜索以减少解码延迟,(2)高效的二维查找表,以及(3)空间-时间混合设计以减少数据缓存,提高查找表的吞吐量。我们开发了一种训练配方,将现有模型转换为支持查找表,具有高精度,并在AMD V80 FPGA上原型化LUT-LLM,用于Qwen 3 1.7B模型,减少算术运算4倍,实现生成速度比GPU快1.10至3.29倍,能效比GPU高3.05至6.60倍。

英文摘要:

The rapid development of large language models (LLM) has greatly enhanced everyday applications. While many FPGA-based accelerators, with flexibility for fine-grained data control, exhibit superior speed and energy efficiency compared to GPUs, recent GPU-specific optimizations have diminished this advantage. When limited to arithmetic-based computation, FPGAs often underperform GPUs due to their comparatively fewer computational resources. To address this challenge, we exploit a key advantage of FPGAs over GPUs: abundant distributed on-chip memory embedded among computational units. We believe that shifting LLM inference from arithmetic-based to memory-based computations through table lookups can improve the efficiency on FPGAs to compete with GPUs. However, existing methods are inefficient or unable to scale and deploy language models due to algorithm and architecture design limitations. This paper introduces \textbf{LUT-LLM}, the first FPGA accelerator that deploy 1B+ language model with memory-based computation, leveraging vector quantization. We construct a performance model, evaluate multiple quantization schemes, and identify activation-weight vector co-quantization as the most effective approach. To support this scheme, LUT-LLM features (1) bandwidth-aware parallel centroid search to reduce decoding latency, (2) efficient 2D table lookups, and (3) a spatial-temporal hybrid design to reduce data caching for a higher throughput table lookup. We develop a training recipe that converts existing models to support table lookups with high accuracy and prototype LUT-LLM for Qwen 3 1.7B model on the AMD V80 FPGA, reducing arithmetic operations by $4\times$ and achieving a $1.10\sim3.29\times$ faster generation speed and a $3.05\sim 6.60\times$ higher energy efficiency than GPUs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑