arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.14852cs.LG
收藏

PV-Tuning:超越直通估计的极限LLM压缩

PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression

Yandex;HSE University(高等经济大学);MIPT(莫斯科物理技术学院);Researchcore;AI Initiative(人工智能倡议);KAUST(阿卜杜拉国王科技大学);Skoltech(斯科尔科沃科技学院);IST Austria(奥地利科学技术研究所);NeuralMagic

Vladimir Malinovskii, Denis Mazur, Ivan Ilin, Denis Kuznedelev, Konstantin Burlachenko, Kai Yi, Dan Alistarh, Peter Richtarik

AI总结:

针对极限LLM压缩中直通估计器(STE)的次优问题,提出与表示无关的PV-Tuning框架,通过改进量化感知微调策略,在1-2比特量化下实现了Llama 2等模型的首个Pareto最优量化。

摘要:

人们对大型语言模型(LLMs)的“极限”压缩(即将每个参数压缩至1-2比特)有着浓厚兴趣,这使得此类模型能在资源受限的设备上高效执行。现有工作侧重于改进一次性量化技术和权重表示;然而,纯训练后方法在精度与比特宽度的权衡上正面临边际收益递减。诸如QuIP#和AQLM等最先进的量化方法包含在少量校准数据上对(部分)压缩参数进行微调;然而,此类基于压缩权重的微调技术通常专门使用直通估计器(STE),其在该场景下的性能尚未被充分理解。在本研究中,我们质疑在极限LLM压缩中使用STE的做法,表明其可能是次优的,并对LLM的量化感知微调策略进行了系统性研究。我们提出了PV-Tuning——一种与表示无关的框架,它泛化并改进了现有的微调策略,并在受限情况下提供了收敛保证。在实践层面,当用于1-2比特向量量化时,PV-Tuning在Llama和Mistral等高性能模型上优于先前技术。借助PV-Tuning,我们首次实现了Llama 2系列模型在每参数2比特下的Pareto最优量化。

英文摘要:

There has been significant interest in "extreme" compression of large language models (LLMs), i.e., to 1-2 bits per parameter, which allows such models to be executed efficiently on resource-constrained devices. Existing work focused on improved one-shot quantization techniques and weight representations; yet, purely post-training approaches are reaching diminishing returns in terms of the accuracy-vs-bit-width trade-off. State-of-the-art quantization methods such as QuIP# and AQLM include fine-tuning (part of) the compressed parameters over a limited amount of calibration data; however, such fine-tuning techniques over compressed weights often make exclusive use of straight-through estimators (STE), whose performance is not well-understood in this setting. In this work, we question the use of STE for extreme LLM compression, showing that it can be sub-optimal, and perform a systematic study of quantization-aware fine-tuning strategies for LLMs. We propose PV-Tuning - a representation-agnostic framework that generalizes and improves upon existing fine-tuning strategies, and provides convergence guarantees in restricted cases. On the practical side, when used for 1-2 bit vector quantization, PV-Tuning outperforms prior techniques for highly-performant models such as Llama and Mistral. Using PV-Tuning, we achieve the first Pareto-optimal quantization for Llama 2 family models at 2 bits per parameter.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑