SAL-PIM：一种面向基于Transformer的文本生成、采用基于LUT的线性插值的子阵列级存内计算架构
SAL-PIM: A Subarray-level Processing-in-Memory Architecture with LUT-based Linear Interpolation for Transformer-based Text Generation
AI总结：
针对基于Transformer的文本生成任务内存瓶颈问题，提出子阵列级存内计算架构SAL-PIM，通过优化数据映射、基于LUT的线性插值等设计实现端到端加速，在GPT-2 medium模型上相较服务器级GPU最高获4.72倍加速。
摘要：
文本生成是自然语言处理中一个极具吸引力的子领域，旨在根据输入词汇生成人类可读的文本。其中，仅解码器架构的生成式模型，例如生成式预训练Transformer（GPT），被广泛应用于文本生成任务，该类模型包含两个主要计算阶段：摘要阶段与生成阶段。与可并行处理输入词元的摘要阶段不同，生成阶段需要通过迭代逐次生成输出词元，因此难以加速。此外，每次迭代都需要读取完整模型，几乎没有数据复用的机会。因此，基于Transformer的文本生成工作负载存在严重的内存瓶颈，外部内存带宽成为系统的性能瓶颈。本文提出了一种名为SAL-PIM的子阵列级存内计算架构，这是一种基于HBM的存内计算架构，用于端到端加速基于Transformer的文本生成。SAL-PIM架构包含三项架构特性：第一，SAL-PIM架构通过集成多个子阵列级算术逻辑单元并搭配优化的数据映射方案，利用更高的内部带宽；第二，SAL-PIM架构采用基于LUT的线性插值在存内计算中实现复杂非线性函数；第三，SAL-PIM架构可在存内计算上加速文本生成的端到端推理。此外，为验证SAL-PIM架构的性能，我们构建了周期精确模拟器，并采用28纳米CMOS工艺实现了SAL-PIM的逻辑单元。实验结果表明，在输入规模为32至128、输出规模为1至256的条件下，基于GPT-2 medium模型的文本生成任务中，SAL-PIM相较于服务器级GPU最高实现4.72倍加速比，平均加速比为1.83倍。
英文摘要：
Text generation is a compelling sub-field of natural language processing, aiming to generate human-readable text from input words. In particular, the decoder-only generative models, such as generative pre-trained transformer (GPT), are widely used for text generation, with two major computational stages: summarization and generation. Unlike the summarization stage, which can process the input tokens in parallel, the generation stage is difficult to accelerate due to its sequential generation of output tokens through iteration. Moreover, each iteration requires reading a whole model with little data reuse opportunity. Therefore, the workload of transformer-based text generation is severely memory-bound, making the external memory bandwidth system bottleneck. In this paper, we proposed a subarray-level processing-in-memory architecture named SAL-PIM, HBM-based PIM architecture for the end-to-end acceleration of transformer-based text generation. The SAL-PIM architecture includes three architectural features. First, the SAL-PIM architecture utilizes higher internal bandwidth by integrating multiple subarray-level arithmetic logic units with optimized data mapping schemes. Second, the SAL-PIM architecture adopts LUT-based linear interpolation to perform complex non-linear functions in PIM. Third, the SAL-PIM architecture accelerates end-to-end inference on PIM in text generation. Furthermore, to validate the SAL-PIM architecture, we built cycle-accurate simulator and implemented the SAL-PIM's logic units in 28-nm CMOS technology. As a result, when the input size is from 32 to 128 and the output size is from 1 to 256, SAL-PIM achieves a maximum of 4.72 times speedup and an average of 1.83 times speedup for the text generation based on the GPT-2 medium model compared to the server-level GPU.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
