S2D：选择性谱衰减用于神经激活的量化友好条件化
S2D: Selective Spectral Decay for Quantization-Friendly Conditioning of Neural Activations
AI总结：
S2D通过选择性谱衰减减少神经激活异常值，提升模型在量化过程中的准确性和部署效率。
摘要：
大规模变换器模型中的激活异常值对模型量化构成了根本性挑战，产生过大的范围，导致量化过程中出现严重的准确性下降。我们实证观察到，异常值的严重性随着预训练规模（例如从CLIP发展到更广泛训练的SigLIP和SigLIP2）而加剧。通过理论分析以及实证相关研究，我们建立了这些激活异常值与权重主导奇异值之间的直接联系。基于这一见解，我们提出了选择性谱衰减（S²D），一种基于几何原理的条件化方法，该方法在微调过程中仅对与最大奇异值对应的权重组件进行手术性正则化。通过广泛的实验，我们证明了S²D显著减少了激活异常值，并产生了固有量化友好的表示。使用S²D训练的模型在ImageNet上实现了在W4A4量化下的高达7%的PTQ准确性提升，并在结合QAT时获得4%的提升。这些改进在下游任务和视觉-语言模型中也具有泛化能力，使能够扩展更大且严格训练的模型，而不会牺牲部署效率。
英文摘要：
Activation outliers in large-scale transformer models pose a fundamental challenge to model quantization, creating excessively large ranges that cause severe accuracy drops during quantization. We empirically observe that outlier severity intensifies with pre-training scale (e.g., progressing from CLIP to the more extensively trained SigLIP and SigLIP2). Through theoretical analysis as well as empirical correlation studies, we establish the direct link between these activation outliers and dominant singular values of the weights. Building on this insight, we propose Selective Spectral Decay ($S^2D$), a geometrically-principled conditioning method that surgically regularizes only the weight components corresponding to the largest singular values during fine-tuning. Through extensive experiments, we demonstrate that $S^2D$ significantly reduces activation outliers and produces well-conditioned representations that are inherently quantization-friendly. Models trained with $S^2D$ achieve up to 7% improved PTQ accuracy on ImageNet under W4A4 quantization and 4% gains when combined with QAT. These improvements also generalize across downstream tasks and vision-language models, enabling the scaling of increasingly large and rigorously trained models without sacrificing deployment efficiency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
