重新思考视觉-语言模型的实用且高效的量化校准
Rethinking Practical and Efficient Quantization Calibration for Vision-Language Models
AI总结：
TLQ通过标记级重要性整合和多GPU层级校准，提升视觉-语言模型的量化稳定性与效率。
摘要：
后训练量化（PTQ）是部署大语言模型而无需微调的主要方法，量化性能往往强烈受到PTQ中的校准影响。相比之下，在视觉-语言模型（VLMs）中，视觉和文本标记在激活分布和对量化误差的敏感性之间存在显著差异，这在PTQ中的有效校准提出了重大挑战。在本工作中，我们重新思考PTQ校准在VLMs中应与什么对齐，并提出Token级重要性感知的层级量化框架（TLQ）。受梯度信息引导，我们设计了一种标记级重要性整合机制用于量化误差，并利用它构建标记级校准集，从而实现更精细的校准策略。此外，TLQ引入了多GPU、暴露于量化的层级校准方案。该方案使层级校准过程与真实的量化推理路径保持一致，并将复杂的层级校准工作量分布到多个RTX3090 GPU上，从而减少对A100 GPU大内存的依赖。TLQ在两个模型、三个模型规模和两种量化设置上进行了评估，始终在所有设置中实现性能提升，表明其强大的量化稳定性。代码将公开发布。
英文摘要：
Post-training quantization (PTQ) is a primary approach for deploying large language models without fine-tuning, and the quantized performance is often strongly affected by the calibration in PTQ. By contrast, in vision-language models (VLMs), substantial differences between visual and text tokens in their activation distributions and sensitivities to quantization error pose significant challenges for effective calibration during PTQ. In this work, we rethink what PTQ calibration should align with in VLMs and propose the Token-level Importance-aware Layer-wise Quantization framework (TLQ). Guided by gradient information, we design a token-level importance integration mechanism for quantization error, and use it to construct a token-level calibration set, enabling a more fine-grained calibration strategy. Furthermore, TLQ introduces a multi-GPU, quantization-exposed layer-wise calibration scheme. This scheme keeps the layer-wise calibration procedure consistent with the true quantized inference path and distributes the complex layer-wise calibration workload across multiple RTX3090 GPUs, thereby reducing reliance on the large memory of A100 GPUs. TLQ is evaluated across two models, three model scales, and two quantization settings, consistently achieving performance improvements across all settings, indicating its strong quantization stability. The code will be released publicly.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
