Outlier Suppression+:通过等效且最优的移位与缩放实现大型语言模型的精确量化
Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling
AI总结:
提出Outlier Suppression+框架,通过等效且最优的逐通道移位与缩放解决Transformer模型激活值异常值问题,在多模型多任务上实现高精度量化并刷新4-bit BERT纪录。
摘要:
由于激活值中存在有害的异常值,Transformer语言模型的训练后量化(PTQ)面临重大挑战。我们观察到这些异常值集中在特定通道中,并且在通道间是不对称的。为了解决这个问题,我们提出了Outlier Suppression+(OS+)框架,其中包含用于处理不对称性的逐通道移位和用于处理集中度的逐通道缩放。我们证明这些操作可以无缝迁移到后续模块中,同时保持等效性。其次,我们提出了一种快速且稳定的方案来计算有效的移位和缩放值。逐通道移位对齐每个通道的中心,以消除异常值的不对称性。逐通道缩放定量评估迁移和量化带来的变化,以更好地平衡量化负担。我们在标准和细粒度量化设置下,使用包括BERT、OPT、BLOOM、BLOOMZ和LLaMA在内的模型验证了我们的OS+。各种任务的综合结果证明了我们方法的优越性。特别是在标准量化下,OS+在8位和6位的小模型和大型语言模型上都能达到接近浮点数的性能。此外,我们为4位BERT建立了新的最先进水平,提升了15.5%。我们的代码可在https://github.com/ModelTC/Outlier_Suppression_Plus获取。
英文摘要:
Post-training quantization~(PTQ) of transformer language models faces significant challenges due to the existence of detrimental outliers in activations. We observe that these outliers are concentrated in specific channels and are asymmetric across channels. To address this issue, we propose the Outlier Suppression+~(OS+) framework, which contains the channel-wise shifting for asymmetry and channel-wise scaling for concentration. We show that these operations can be seamlessly migrated into subsequent modules while maintaining equivalence. Second, we propose a fast and stable scheme to calculate effective shifting and scaling values. The channel-wise shifting aligns the center of each channel for removal of outlier asymmetry. The channel-wise scaling quantitatively evaluates changes brought by migration and quantization for better quantization burden balance. We validate our OS+ under both standard and fine-grained quantization settings with models including BERT, OPT, BLOOM, BLOOMZ, and LLaMA. Comprehensive results across various tasks demonstrate the superiority of our approach. Especially, with standard quantization, OS+ can achieve near-floating-point performance on both small models and large language models on 8-bit and 6-bit. Besides, we establish a new state-of-the-art for 4-bit BERT with 15.5\% improvement. Our code is available at \url{https://github.com/ModelTC/Outlier_Suppression_Plus}.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。