ZeroQuant(4+2):以新型FP6为核心的策略重新定义大语言模型量化,适用于多样化生成任务
ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Tasks
AI总结:
针对现有4位量化在生成类任务上性能下降、FP6因硬件适配难被忽视的问题,提出FP6 4+2设计,在保证精度与通用性的同时实现与INT4相当的延迟,为LLM量化提供新方案。
摘要:
本研究考察了GPTQ等大语言模型(LLM)的4位量化方法,指出GPTQ存在过拟合问题,且在零样本(Zero-Shot)任务上的提升有限。以往研究仅聚焦零样本指标,我们将任务范围拓展至代码生成、抽象摘要等更多生成类任务,发现INT4量化在这些任务上的性能会显著下降。然而,直接转向FP6等更高精度格式的难度极大,因此长期被忽视——原因在于当前AI硬件缺乏成熟的集成与系统加速策略,导致FP6性能不佳。我们的实验结果表明,即便采用粗粒度量化方案,FP6在各类算法和任务上均表现稳健,在准确性和通用性上更具优势。值得注意的是,经FP6量化后,\u200bcodestar-15B模型在代码生成任务上的性能与FP16版本相当;对于406M这类更小的模型,其在摘要任务上的表现也与基线模型接近,这两点都是INT4无法实现的。为适配不同AI硬件并实现最优系统性能,我们提出了一种新颖的FP6 4+2设计,其延迟可与当前最先进的INT4细粒度量化持平。通过该设计,FP6有望成为现有LLM 4位量化方法的理想替代方案。
英文摘要:
This study examines 4-bit quantization methods like GPTQ in large language models (LLMs), highlighting GPTQ's overfitting and limited enhancement in Zero-Shot tasks. While prior works merely focusing on zero-shot measurement, we extend task scope to more generative categories such as code generation and abstractive summarization, in which we found that INT4 quantization can significantly underperform. However, simply shifting to higher precision formats like FP6 has been particularly challenging, thus overlooked, due to poor performance caused by the lack of sophisticated integration and system acceleration strategies on current AI hardware. Our results show that FP6, even with a coarse-grain quantization scheme, performs robustly across various algorithms and tasks, demonstrating its superiority in accuracy and versatility. Notably, with the FP6 quantization, \codestar-15B model performs comparably to its FP16 counterpart in code generation, and for smaller models like the 406M it closely matches their baselines in summarization. Neither can be achieved by INT4. To better accommodate various AI hardware and achieve the best system performance, we propose a novel 4+2 design for FP6 to achieve similar latency to the state-of-the-art INT4 fine-grain quantization. With our design, FP6 can become a promising solution to the current 4-bit quantization methods used in LLMs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。