发表机构
National Yang Ming Chiao Tung University（国立阳明交通大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文针对低比特量化后训练方法中门控机制误删可补偿层的问题，提出无参数截断伪逆求解器，在ViT-B模型上实现了优于现有方法的精度，且在模型规模与精度间取得良好平衡。
AI 中文摘要
近期，无训练的后训练量化方法通过闭式残差补偿恢复模型精度。为约束额外的模型存储开销，现有若干方法通过拟合优度进行层选择，仅保留那些补偿后残差拟合得分为正的层，丢弃其余层。本文表明，在低比特W4A4设置下，该门控机制无法区分难以预测的量化误差与数值求解器故障。秩亏的输入激活会产生严重病态或数值奇异的Gram矩阵，导致闭式求解器不稳定并产生虚假的负拟合得分。因此，现有的拟合优度门将受影响的层误分类为不可补偿层并丢弃它们。然而，许多此类被丢弃的层在使用数值稳定求解器计算其补偿时，仍可提供显著的误差恢复效果。为解决该问题，我们提出一种无参数的截断伪逆求解器，其在求逆前移除坍缩方向。在采用W4A4设置的ViT-B模型上，我们的无训练方法达到了81.42%的top-1准确率，优于现有的后训练方法和基于微调的基线。结合联合低秩与量化压缩，该方法在54.7 MB的存储下达到了80.26%的可部署操作点，在模型规模与精度间实现了良好的平衡。
英文摘要
Recent training-free post-training quantization methods restore model accuracy through closed-form residual compensation. To constrain additional model storage overhead, several existing methods gate layer selection by goodness-of-fit, retaining only those layers whose compensation yields a positive residual fit score and discarding the rest. In this paper, we show that, under the low-bit W4A4 setting, this gating mechanism fails to distinguish poorly predictable quantization error from numerical solver failure. Rank-deficient input activations yield severely ill-conditioned or numerically singular Gram matrices, causing the closed-form solver to become unstable and produce spuriously negative fit scores. Consequently, existing goodness-of-fit gates misclassify affected layers as uncompensable and discard them. Many of these discarded layers can nevertheless provide substantial error recovery when their compensation is computed using a numerically stable solver. To address this problem, we propose a parameter-free truncated pseudoinverse solver which removes collapsed directions prior to inversion. On ViT-B with the W4A4 setting, our training-free method achieves 81.42\% top-1 accuracy, outperforming prior post-training methods and fine-tuning-based baselines. Combined with joint low-rank and quantization compression, the proposed method reaches a deployable operating point of 80.26\% accuracy at 54.7 MB, providing a well-balanced trade-off between model size and accuracy.
Comments8 pages, 6 figures
