OPAL：面向生成式大语言模型的保留异常值微缩放量化加速器
OPAL: Outlier-Preserved Microscaling Quantization Accelerator for Generative Large Language Models
AI总结：
本文提出软硬件协同设计的LLM加速器OPAL，通过保留异常值的微缩放激活量化、混合精度及优化硬件架构，在精度几乎无损下显著提升能效并减少面积。
摘要：
为了克服大语言模型（LLMs）规模不断增大带来的内存大小和带宽负担，近期人们研究了激进的权重量化，但缺乏对激活量化的研究。在本文中，我们提出了一种软硬件协同设计方法，实现了一个名为 OPAL 的能效极高的 LLM 加速器，用于生成任务。首先，提出了一种新颖的激活量化方法，该方法利用微缩放数据格式，同时保留每个子张量块中的几个异常值（例如，128 个元素中的 4 个）。其次，在保留异常值的基础上，利用了混合精度，将 LLM 解码器块中敏感层的输入设置为 5-bit，同时将不敏感层的输入保持为 3-bit。最后，我们提出了 OPAL 硬件架构，该架构由用于处理异常值的 FP 单元和用于主导非异常值相关操作的向量化 INT 乘法器组成。此外，OPAL 在 softmax 操作上使用基于 log2 的近似，仅需要移位和减法来最大化能效。结果，我们能够将能效提高 1.6~2.2 倍，并将面积减少 2.4~3.1 倍，且精度损失可忽略不计，即困惑度增加 <1。
英文摘要：
To overcome the burden on the memory size and bandwidth due to ever-increasing size of large language models (LLMs), aggressive weight quantization has been recently studied, while lacking research on quantizing activations. In this paper, we present a hardware-software co-design method that results in an energy-efficient LLM accelerator, named OPAL, for generation tasks. First of all, a novel activation quantization method that leverages the microscaling data format while preserving several outliers per sub-tensor block (e.g., four out of 128 elements) is proposed. Second, on top of preserving outliers, mixed precision is utilized that sets 5-bit for inputs to sensitive layers in the decoder block of an LLM, while keeping inputs to less sensitive layers to 3-bit. Finally, we present the OPAL hardware architecture that consists of FP units for handling outliers and vectorized INT multipliers for dominant non-outlier related operations. In addition, OPAL uses log2-based approximation on softmax operations that only requires shift and subtraction to maximize power efficiency. As a result, we are able to improve the energy efficiency by 1.6~2.2x, and reduce the area by 2.4~3.1x with negligible accuracy loss, i.e., <1 perplexity increase.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
