MASQuant: 多模态大语言模型的模态感知平滑量化
MASQuant: Modality-Aware Smoothing Quantization for Multimodal Large Language Models
AI总结:
MASQuant通过模态感知平滑和跨模态补偿技术,解决多模态大语言模型的量化问题,实现稳定且高效的量化性能。
摘要:
后训练量化(PTQ)在大语言模型(LLMs)中已展现出显著进展,然而将其应用于多模态大语言模型(MLLMs)则面临重大挑战。本文以SmoothQuant为案例研究,识别出两个关键问题:平滑偏移和跨模态计算不变性。为解决这些问题,我们提出了模态感知平滑量化(MASQuant),一种新的框架,引入(1)模态感知平滑(MAS),通过学习独立的、模态特定的平滑因子来防止平滑偏移,以及(2)跨模态补偿(CMC),通过使用SVD白化将多模态激活差异转换为低秩形式,从而实现跨模态的统一量化。MASQuant在双模态和三模态MLLMs上均表现出稳定的量化性能。实验结果表明,MASQuant在最先进的PTQ算法中具有竞争力。源代码:https://github.com/alibaba/EfficientAI。
英文摘要:
Post-training quantization (PTQ) with computational invariance for Large Language Models~(LLMs) have demonstrated remarkable advances, however, their application to Multimodal Large Language Models~(MLLMs) presents substantial challenges. In this paper, we analyze SmoothQuant as a case study and identify two critical issues: Smoothing Misalignment and Cross-Modal Computational Invariance. To address these issues, we propose Modality-Aware Smoothing Quantization (MASQuant), a novel framework that introduces (1) Modality-Aware Smoothing (MAS), which learns separate, modality-specific smoothing factors to prevent Smoothing Misalignment, and (2) Cross-Modal Compensation (CMC), which addresses Cross-modal Computational Invariance by using SVD whitening to transform multi-modal activation differences into low-rank forms, enabling unified quantization across modalities. MASQuant demonstrates stable quantization performance across both dual-modal and tri-modal MLLMs. Experimental results show that MASQuant is competitive among the state-of-the-art PTQ algorithms. Source code: https://github.com/alibaba/EfficientAI.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。