KBBQ：预测性噪声定律与FP4量化中频谱展平的极限
KBBQ: A Predictive Noise Law and the Limits of Spectrum Flattening in FP4 Quantization
浏览论文内容
中文总结 AI 辅助
KBBQ提出量化噪声二阶理论，推导信噪比定律及上界，并据此设计分块量化方法，在W4A4下超越现有最先进方法。
中文摘要 AI 辅助
我们发展了矩阵乘法中量化噪声的二阶理论，其中量化格式通过其分配给每个元素的方差来表征。整数量化的常数方差分布恢复了现有的整数噪声理论，而浮点舍入的乘法分布将数据依赖性简化为一个标量，即参与因子 $\kappa$，从而产生一个闭式的信噪比定律。由此得到的泛函还允许一个闭式上界 $\kappa^{*}$，任何保持函数的线性变换都无法超过该上界，并且该上界由最近的一种最先进方法所达到。基于这一分析，我们引入了KBBQ（$\kappa$约束分块量化），它参数化了变换接近这一上限的程度。在W4A4设置下，跨四个基础模型和两种FP4格式，KBBQ在无需额外部署时计算的情况下优于先前的最先进方法。
英文摘要
We develop a second-order theory of quantization noise in matrix multiplication in which the quantization format is characterized by the variance it assigns to each element. The constant variance profile of integer quantization recovers existing integer-noise theory, while the multiplicative profile of floating-point rounding reduces the data dependence to a scalar, the participation factor $κ$, yielding a closed-form signal-to-noise-ratio law. The resulting functional also admits a closed-form upper bound $κ^{*}$ that no function-preserving linear transform can exceed and that is attained by a recent state-of-the-art method. Building on this analysis, we introduce KBBQ (\textbf{K}appa-\textbf{B}raked \textbf{B}lockwise \textbf{Q}uantization), which parameterizes the extent to which a transform approaches this ceiling. At W4A4, across four base models and two FP4 formats, KBBQ outperforms the prior state of the art without additional deployment-time computation.
发表机构
- SB Intuitions
机构由 AI 辅助整理，请以论文原文为准。
