Activation Outliers in Transformer Quantization: Reproduction, Statistical Analysis, and Deployment Tradeoffs
Transformer量化中的激活异常:重现、统计分析与部署权衡
机构 * University of Colorado Denver(科罗拉多大学丹佛分校)
AI总结 本文研究了Transformer量化中的结构化激活异常问题,通过实验发现量化导致精度大幅下降,并提出混合精度和按嵌入组量化等缓解方法,强调通道意识精度分配的重要性。
Comments 10 pages, 3 tables. Reproducible study of transformer PTQ activation outliers based on Bondarenko et al. (EMNLP 2021, Qualcomm AI Research). Code: https://github.com/pranavkkp4/TransQuant-Edge