SandwichQuant：量化前后哪些参数起作用？
SandwichQuant: Which Parameters Matter Before and After Quantization?
- School of Information Science and Technology, Beijing University of Technology（北京工业大学信息科学与技术学院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究提出SandwichQuant框架，发现归一化仿射参数子空间是量化校正的关键，该两阶段框架在视觉与大语言模型的低比特量化中均实现了性能提升。
AI中文摘要：
量化校正方法通常会优化权重、量化参数或重构目标，但负责有效校正的底层参数子空间仍不明确。本研究从参数子空间视角研究量化校正，发现不同参数组的校正能力高度不均匀。通过将可训练参数分解为骨干权重、归一化仿射参数和量化参数，我们证明在匹配预算下，低维归一化仿射子空间提供了极具效率的校正方向。基于此发现，我们提出SandwichQuant，这是一个两阶段归一化仿射校正框架，在量化前后均执行适配：前一阶段提升量化鲁棒性，后一阶段在量化图固定后补偿残差误差。在视觉模型和大语言模型上开展的大量实验，在各种低比特量化设置下均展现出一致的性能提升，验证了子空间对齐校正的有效性。
英文摘要：
Quantization correction methods usually optimize weights, quantization parameters, or reconstruction objectives, while the underlying parameter subspaces responsible for effective correction remain unclear. In this work, we study quantization correction from a parameter subspace perspective and reveal that correction capability is highly non-uniform across parameter groups. By decomposing trainable parameters into backbone weights, normalization-affine parameters, and quantization parameters, we show that the low-dimensional normalization-affine subspace provides a highly efficient correction direction under matched budgets. Based on this finding, we propose SandwichQuant, a two-stage normalization-affine correction framework that performs adaptation before and after quantization. The pre-stage improves quantization robustness, while the post-stage compensates residual errors after the quantized graph is fixed. Extensive experiments on vision models and large language models demonstrate consistent improvements under various low-bit quantization settings, validating the effectiveness of subspace-aligned correction.
