TORQ：MXFP4量化中的两级正交旋转
TORQ: Two-Level Orthogonal Rotation for MXFP4 Quantization
AI总结：
本文提出TORQ框架，通过优化坐标变换重塑激活空间的几何属性，解决MXFP4激活量化中的精度下降问题，显著提升量化精度。
摘要：
随着大型语言模型（LLMs）向实际部署迈进，微缩FP4（MXFP4）格式已成为下一代低比特推断的基石，因其在高动态范围与硬件效率之间的平衡能力。然而，直接将MXFP4应用于LLM激活量化不可避免地导致显著的精度下降。在本文中，我们从理论上分析MXFP4激活量化的误差结构，揭示出性能下降的根本原因在于激活分布与MXFP4块浮点格式之间的两个结构性不平衡：（1）极端块间方差不平衡和（2）块内代码书利用不平衡。为了解决这些挑战，我们提出了TORQ（MXFP4量化中的两级正交旋转），一种无训练的后训练量化（PTQ）框架，通过最优坐标变换重塑激活空间的几何属性。在宏观层面，TORQ利用Schur-Horn定理通过块间正交旋转重新分配激活能量，防止高方差块驱动共享缩放因子，从而保留小幅度元素的精度。在微观层面，TORQ采用最大熵引导的块内旋转以缓解代码书坍塌并最大化MXFP4代码书的信息容量。在主流LLM如LLaMA3和Qwen3上的实验表明，与现有方法相比，TORQ显著提高了MXFP4激活量化的准确性：在Qwen3-32B上，WikiText的困惑度降低到8.43（相比BF16的7.61），平均准确率从直接RTN的38.40%增加到73.63%（相比BF16的74.82%），大幅缩小了4位浮点量化与全精度推断之间的差距。
英文摘要：
As Large Language Models (LLMs) advance toward practical deployment, the Microscaling FP4 (MXFP4) format has emerged as a cornerstone for next-generation low-bit inference, owing to its ability to balance high dynamic range with hardware efficiency. However, directly applying MXFP4 to LLM activation quantization inevitably leads to significant accuracy degradation. In this paper, we theoretically analyze the error structure of MXFP4 activation quantization, revealing that the root cause of this performance drop lies in two structural imbalances between activation distributions and the MXFP4 block floating-point format: (1) extreme inter-block variance imbalance and (2) intra-block codebook utilization imbalance. To address these challenges, we propose TORQ (Two-level Orthogonal Rotation for MXFP4 Quantization), a training-free Post-Training Quantization (PTQ) framework designed to reshape the geometric properties of the activation space through optimal coordinate transformations. At the macroscopic level, TORQ leverages the Schur-Horn theorem to redistribute activation energy via inter-block orthogonal rotation, preventing high-variance blocks from driving up shared scaling factors and thereby preserving the precision of small-magnitude elements. At the microscopic level, TORQ employs maximum-entropy-guided intra-block rotation to alleviate codebook collapse and maximize the MXFP4 codebook's information capacity. Experiments on mainstream LLMs such as LLaMA3 and Qwen3 show that TORQ significantly improves the accuracy of MXFP4 activation quantization compared to existing methods: on Qwen3-32B, the perplexity on WikiText is reduced to 8.43 (vs. 7.61 for BF16), and the average accuracy increases from 38.40% with direct RTN to 73.63% (vs. 74.82% for BF16), substantially narrowing the gap between 4-bit floating-point quantization and full-precision inference.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
