RepQuant：通过尺度重参数化实现大型Transformer模型的精确训练后量化
RepQuant: Towards Accurate Post-Training Quantization of Large Transformer Models via Scale Reparameterization
AI总结：
RepQuant提出量化-推理解耦范式，通过尺度重参数化在训练后量化中兼顾精度与硬件兼容性，显著提升大型Transformer模型的压缩性能。
摘要：
大型Transformer模型已展现出显著的成功。训练后量化（PTQ）仅需少量数据集进行校准，且无需端到端重新训练，是压缩这些大型模型的一种有前景的解决方案。遗憾的是，现有的PTQ方法通常会表现出不可忽视的性能损失。我们发现，性能瓶颈源于量化过程中对硬件兼容性的过度考虑，这迫使它们勉强采用简单的量化器，尽管以牺牲精度为代价。基于上述见解，我们提出了RepQuant，一种新颖的PTQ框架，采用量化-推理解耦范式来解决上述问题。RepQuant在量化过程中使用复杂的量化器，在推理过程中使用简化的量化器，并通过量化尺度重参数化在两者之间执行数学上等价的变换，从而确保既精确量化又高效推理。更具体地，我们关注两个具有极端分布的组件：LayerNorm激活和Softmax激活。首先，我们分别应用通道级量化和log√2量化，这些量化针对其分布量身定制。特别地，对于前者，我们引入了一种可学习的逐通道双重裁剪方案，旨在以细粒度高效识别不平衡激活中的异常值。然后，我们将尺度重参数化为硬件友好的逐层量化和log2量化以用于推理。此外，量化权重重建无缝集成到上述过程中，以进一步推动性能极限。在多个任务上对不同的规模化Transformer变体进行了广泛实验，包括视觉、语言和多模态Transformer，RepQuant令人鼓舞地展示了显著的性能优势。
英文摘要：
Large transformer models have demonstrated remarkable success. Post-training quantization (PTQ), which requires only a small dataset for calibration and avoids end-to-end retraining, is a promising solution for compressing these large models. Regrettably, existing PTQ methods typically exhibit non-trivial performance loss. We find that the performance bottleneck stems from over-consideration of hardware compatibility in the quantization process, compelling them to reluctantly employ simple quantizers, albeit at the expense of accuracy. With the above insights, we propose RepQuant, a novel PTQ framework with quantization-inference decoupling paradigm to address the above issues. RepQuant employs complex quantizers in the quantization process and simplified quantizers in the inference process, and performs mathematically equivalent transformations between the two through quantization scale reparameterization, thus ensuring both accurate quantization and efficient inference. More specifically, we focus on two components with extreme distributions: LayerNorm activations and Softmax activations. Initially, we apply channel-wise quantization and log$\sqrt{2}$ quantization, respectively, which are tailored to their distributions. In particular, for the former, we introduce a learnable per-channel dual clipping scheme, which is designed to efficiently identify outliers in the unbalanced activations with fine granularity. Then, we reparameterize the scales to hardware-friendly layer-wise quantization and log2 quantization for inference. Moreover, quantized weight reconstruction is seamlessly integrated into the above procedure to further push the performance limits. Extensive experiments are performed on different large-scale transformer variants on multiple tasks, including vision, language, and multi-modal transformers, and RepQuant encouragingly demonstrates significant performance advantages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
