RaZeR: 推动NVFP4量化极限的冗余零重映射
RaZeR: Pushing the Limits of NVFP4 Quantization with Redundant Zero Remapping
AI总结：
RaZeR通过冗余零重映射提升NVFP4量化精度，降低LLM推理的困惑度损失。
摘要：
最近引入的NVFP4格式在量化大语言模型（LLM）推理中展现出显著的性能和内存优势。然而，我们观察到NVFP4编码中存在两种冗余：（1）FP4元素格式由于其具有正负零的符号幅度表示，自然暴露了一个未使用的量化值。（2）FP8块缩放因子有一个未使用的符号位，因为它始终为正。此外，我们发现LLM权重对较低精度的块缩放因子更具容忍性。基于这些观察，我们提出了冗余零重映射（RaZeR），一种增强的数值格式，旨在在相同内存占用下推动NVFP4的极限，以实现更准确的LLM量化。RaZeR利用块缩放因子的冗余位，自适应地将冗余的FP4零映射到额外的量化值，以提高精度。为了证明RaZeR的实用性，我们设计了高效的GPU内核用于RaZeR量化LLM推理，并提出了新的硬件以原生支持此功能。广泛的实验验证了RaZeR在4位LLM量化中的优越性能。例如，与原生NVFP4相比，RaZeR在权重仅量化和权重-激活量化下分别将平均困惑度损失降低了34.6%和31.2%。代码可在：https://github.com/yc2367/NVFP4-RaZeR获取。
英文摘要：
The recently introduced NVFP4 format demonstrates remarkable performance and memory benefits for quantized large language model (LLM) inference. However, we observe two types of redundancy in NVFP4 encoding: (1) The FP4 element format naturally exposes an unused quantization value due to its sign-magnitude representation that contains both positive and negative zeros. (2) The FP8 block scaling factor has an unused sign bit because it is always positive. Additionally, we find that LLM weights are more tolerant to a lower-precision block scaling factor. Based on these observations, we propose Redundant Zero Remapping (RaZeR), an enhanced numerical format that pushes the limits of NVFP4 for more accurate LLM quantization under the same memory footprint. RaZeR leverages the redundant bits of the block scaling factor to adaptively remap the redundant FP4 zero to additional quantization values with improved accuracy. To demonstrate the practicality of RaZeR, we design efficient GPU kernels for RaZeR-quantized LLM inference and propose novel hardware to natively support this. Extensive experiments validate RaZeR's superior performance for 4-bit LLM quantization. For example, relative to native NVFP4, RaZeR reduces the average perplexity loss by 34.6% and 31.2% under weight-only and weight-activation quantization, respectively. Code is available at: https://github.com/yc2367/NVFP4-RaZeR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
