arXiv 2407.027632024-07cs.CV

ADFQ-ViT：面向视觉Transformer的激活分布友好的训练后量化

ADFQ-ViT: Activation-Distribution-Friendly Post-Training Quantization for Vision Transformers

College of Computer Science, Nankai University（南开大学计算机学院）；Zhejiang Lab（之江实验室）；Haihe Lab of ITAI（海河人工智能研究院）

Yanfeng Jiang, Ning Sun, Xueshuo Xie, Fei Yang, Tao Li

备注：: 29 pages,9 figures

AI总结：

针对ViTs在低比特量化下因post-LayerNorm和post-GELU激活分布异常导致精度损失的问题，提出ADFQ-ViT框架，引入Per-Patch Outlier-aware和Shift-Log2量化器及模块级优化，在4-bit下显著提升多项视觉任务的准确率。

摘要：

视觉Transformer（ViTs）在各类计算机视觉任务中展现出卓越的性能，但其庞大的参数量导致内存和计算需求显著增加，阻碍了在资源受限设备上的有效推理。量化已成为缓解这些挑战的一种有前景的解决方案，但现有方法在低比特位下仍存在显著的精度损失。我们将此问题归因于ViTs中post-LayerNorm和post-GELU激活的独特分布，这使得传统的硬件友好型量化器失效，尤其是在低比特场景下。为解决此问题，我们提出了一种名为面向视觉Transformer的激活分布友好型训练后量化（ADFQ-ViT）的新框架。具体而言，我们引入了Per-Patch Outlier-aware Quantizer来处理post-LayerNorm激活中的不规则异常值。该量化器将均匀量化器的粒度细化到per-patch级别，同时将超过阈值的极小部分值保留为全精度。为处理post-GELU激活在正负区域之间的非均匀分布，我们设计了Shift-Log2 Quantizer，它将所有元素移位至正区域，然后应用log2量化。此外，我们提出了Attention-score enhanced Module-wise Optimization，通过重建误差来调整每个量化器的参数，以进一步缓解量化误差。大量实验表明，在4-bit下，ADFQ-ViT在图像分类、目标检测和实例分割任务中相较于各种基线提供了显著的改进。具体而言，当将ViT-B模型量化至4-bit时，我们在ImageNet数据集上的Top-1准确率实现了10.23%的提升。

英文摘要：

Vision Transformers (ViTs) have exhibited exceptional performance across diverse computer vision tasks, while their substantial parameter size incurs significantly increased memory and computational demands, impeding effective inference on resource-constrained devices. Quantization has emerged as a promising solution to mitigate these challenges, yet existing methods still suffer from significant accuracy loss at low-bit. We attribute this issue to the distinctive distributions of post-LayerNorm and post-GELU activations within ViTs, rendering conventional hardware-friendly quantizers ineffective, particularly in low-bit scenarios. To address this issue, we propose a novel framework called Activation-Distribution-Friendly post-training Quantization for Vision Transformers, ADFQ-ViT. Concretely, we introduce the Per-Patch Outlier-aware Quantizer to tackle irregular outliers in post-LayerNorm activations. This quantizer refines the granularity of the uniform quantizer to a per-patch level while retaining a minimal subset of values exceeding a threshold at full-precision. To handle the non-uniform distributions of post-GELU activations between positive and negative regions, we design the Shift-Log2 Quantizer, which shifts all elements to the positive region and then applies log2 quantization. Moreover, we present the Attention-score enhanced Module-wise Optimization which adjusts the parameters of each quantizer by reconstructing errors to further mitigate quantization error. Extensive experiments demonstrate ADFQ-ViT provides significant improvements over various baselines in image classification, object detection, and instance segmentation tasks at 4-bit. Specifically, when quantizing the ViT-B model to 4-bit, we achieve a 10.23% improvement in Top-1 accuracy on the ImageNet dataset.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文