arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.02775cs.AIcs.LG
收藏

EasyQuant:一种面向大语言模型的高效无数据量化算法

EasyQuant: An Efficient Data-free Quantization Algorithm for LLMs

Tencent(腾讯)

Hanlin Tang, Yifu Sun, Decheng Wu, Kai Liu, Jianchen Zhu, Zhanhui Kang

AI总结:

针对现有LLM量化方法依赖训练数据影响泛化的问题,提出无需训练、无数据依赖的EasyQuant纯权重量化算法,通过保留少量离群值、优化量化范围实现近无损量化,速度超数据依赖方法10倍。

摘要:

大语言模型(LLM)已被证实在各类任务中显著优于传统方法,但其高昂的计算成本与内存需求给部署带来了极大阻碍。模型量化是降低这类开销的有效手段,但过往多数研究需使用少量训练数据样本校准量化模型,这可能会影响量化后LLM对未知场景与任务的泛化能力。因此,本研究聚焦一个关键问题:能否设计一种不依赖数据的LLM量化方法,以保障其泛化性能?\n本文提出EasyQuant,这是一种无需训练、不依赖数据的LLM纯权重量化算法。研究观察发现,权重中的离群值与量化范围是降低量化误差的两个核心因素。为此,EasyQuant保留占比不足1%的离群值不变,同时优化量化范围以降低重构误差。通过这些策略,研究意外发现EasyQuant可取得与原始模型相当的性能。由于EasyQuant不依赖任何训练数据,量化后LLM的泛化性能得到了可靠保障。此外,EasyQuant支持并行实现,即便针对参数量超过100B的大语言模型,也能在数分钟内完成量化。据我们所知,本研究是首个在无数据设置下实现LLM近无损量化性能的工作,且算法运行速度比依赖数据的方法快10倍以上。

英文摘要:

Large language models (LLMs) have proven to be very superior to conventional methods in various tasks. However, their expensive computations and high memory requirements are prohibitive for deployment. Model quantization is an effective method for reducing this overhead. The problem is that in most previous works, the quantized model was calibrated using few samples from the training data, which might affect the generalization of the quantized LLMs to unknown cases and tasks. Hence in this work, we explore an important question: Can we design a data-independent quantization method for LLMs to guarantee its generalization performance? In this work, we propose EasyQuant, a training-free and data-independent weight-only quantization algorithm for LLMs. Our observation indicates that two factors: outliers in the weight and quantization ranges, are essential for reducing the quantization error. Therefore, in EasyQuant, we leave the outliers (less than 1%) unchanged and optimize the quantization range to reduce the reconstruction error. With these methods, we surprisingly find that EasyQuant achieves comparable performance to the original model. Since EasyQuant does not depend on any training data, the generalization performance of quantized LLMs is safely guaranteed. Moreover, EasyQuant can be implemented in parallel so that the quantized model could be attained in a few minutes even for LLMs over 100B. To our best knowledge, we are the first work that achieves almost lossless quantization performance for LLMs under a data-independent setting and our algorithm runs over 10 times faster than the data-dependent methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑