面向 Vision Transformers 精确低比特训练后量化的渐进式由细到粗重构
Progressive Fine-to-Coarse Reconstruction for Accurate Low-Bit Post-Training Quantization in Vision Transformers
AI总结：
针对ViTs低比特PTQ性能下降及现有重构方法粒度固定的次优问题，提出渐进式由细到粗重构方法PFCR及优化策略POS，在ImageNet和COCO上取得SOTA精度并验证了泛化性。
摘要：
由于其高效性，Post-Training Quantization (PTQ) 被广泛用于压缩 Vision Transformers (ViTs)。然而，当量化为低比特表示时，与其全精度对应物相比通常会出现显著的性能下降。为了解决这个问题，重构方法已被引入到 PTQ 框架中，以提升低比特量化设置下的性能。尽管如此，现有的相关方法预定义了重构粒度，很少探索不同重构粒度之间的渐进关系，这导致 ViTs 中的量化结果次优。为此，在本文中，我们提出了一种用于精确 PTQ 的 Progressive Fine-to-Coarse Reconstruction (PFCR) 方法，该方法显著提升了低比特量化视觉变换器的性能。具体而言，我们将多头自注意力和多层感知机模块及其快捷连接定义为最细的重构单元。在重构这两个细粒度单元后，我们将它们组合成更粗的块，并在更粗的粒度级别上对其进行重构。我们迭代地执行这一组合和重构过程，实现渐进式的由细到粗重构。此外，我们为 PFCR 引入了 Progressive Optimization Strategy (POS) 以缓解训练难度，从而进一步提升模型性能。在 ImageNet 数据集上的实验结果表明，我们提出的方法在现有最先进方法中实现了最佳的 Top-1 准确率，特别是在 PTQ 中 3 比特量化的 ViT-B 上达到了 75.61%。此外，在 COCO 数据集上的量化结果揭示了我们所提方法在目标检测和实例分割等其他计算机视觉任务上的有效性和泛化能力。
英文摘要：
Due to its efficiency, Post-Training Quantization (PTQ) has been widely adopted for compressing Vision Transformers (ViTs). However, when quantized into low-bit representations, there is often a significant performance drop compared to their full-precision counterparts. To address this issue, reconstruction methods have been incorporated into the PTQ framework to improve performance in low-bit quantization settings. Nevertheless, existing related methods predefine the reconstruction granularity and seldom explore the progressive relationships between different reconstruction granularities, which leads to sub-optimal quantization results in ViTs. To this end, in this paper, we propose a Progressive Fine-to-Coarse Reconstruction (PFCR) method for accurate PTQ, which significantly improves the performance of low-bit quantized vision transformers. Specifically, we define multi-head self-attention and multi-layer perceptron modules along with their shortcuts as the finest reconstruction units. After reconstructing these two fine-grained units, we combine them to form coarser blocks and reconstruct them at a coarser granularity level. We iteratively perform this combination and reconstruction process, achieving progressive fine-to-coarse reconstruction. Additionally, we introduce a Progressive Optimization Strategy (POS) for PFCR to alleviate the difficulty of training, thereby further enhancing model performance. Experimental results on the ImageNet dataset demonstrate that our proposed method achieves the best Top-1 accuracy among state-of-the-art methods, particularly attaining 75.61% for 3-bit quantized ViT-B in PTQ. Besides, quantization results on the COCO dataset reveal the effectiveness and generalization of our proposed method on other computer vision tasks like object detection and instance segmentation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
