TSPTQ-ViT：面向视觉Transformer的双尺度训练后量化
TSPTQ-ViT: Two-scaled post-training quantization for vision transformer
AI总结：
针对ViT训练后量化中Softmax/GeLU非正态分布及LayerNorm高方差导致的精度下降问题，提出双尺度量化方案TSPTQ-ViT，设计V-2SF和O-2SF因子，在8位全量化下实现近乎无损的ImageNet分类精度。
摘要：
视觉Transformer（ViTs）在多种计算机视觉任务中取得了显著性能。然而，密集的内存和计算需求阻碍了ViTs在资源受限的边缘设备上运行。由于Softmax和GeLU之后的值呈非正态分布，ViTs的训练后量化会导致严重的精度下降。此外，传统方法无法处理LayerNorm中较高的通道间方差。为减少量化损失并提高分类精度，我们提出了一种面向视觉Transformer的双尺度训练后量化方案（TSPTQ-ViT）。我们设计了专门针对post-Softmax和post-GeLU值的值感知双尺度缩放因子（V-2SF），利用非正态分布中的比特稀疏性来节省比特宽度。此外，向LayerNorm引入了离群值感知双尺度缩放因子（O-2SF），以缓解离群值的主导影响。实验结果表明，所提方法在8位全量化ViTs的ImageNet分类任务上实现了近乎无损的精度下降（<0.5%）。
英文摘要：
Vision transformers (ViTs) have achieved remarkable performance in various computer vision tasks. However, intensive memory and computation requirements impede ViTs from running on resource-constrained edge devices. Due to the non-normally distributed values after Softmax and GeLU, post-training quantization on ViTs results in severe accuracy degradation. Moreover, conventional methods fail to address the high channel-wise variance in LayerNorm. To reduce the quantization loss and improve classification accuracy, we propose a two-scaled post-training quantization scheme for vision transformer (TSPTQ-ViT). We design the value-aware two-scaled scaling factors (V-2SF) specialized for post-Softmax and post-GeLU values, which leverage the bit sparsity in non-normal distribution to save bit-widths. In addition, the outlier-aware two-scaled scaling factors (O-2SF) are introduced to LayerNorm, alleviating the dominant impacts from outlier values. Our experimental results show that the proposed methods reach near-lossless accuracy drops (<0.5%) on the ImageNet classification task under 8-bit fully quantized ViTs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
