decoupleQ：通过将参数解耦为整数和浮点数实现2比特训练后均匀量化
decoupleQ: Towards 2-bit Post-Training Uniform Quantization via decoupling Parameters into Integer and Floating Points
AI总结：
本文提出decoupleQ，通过将模型参数解耦为整数和浮点数，将量化转化为约束优化问题，实现2比特均匀量化，在字节跳动大型语音模型上达到接近fp16/bf16的精度。
摘要：
量化近年来成为在各种实时应用中部署高效大模型最有前景的压缩技术之一。考虑到在大模型中，权重的存储和输入输出占用了绝大部分开销，仅权重量化可以带来巨大的收益。然而，现有的量化方案在极低比特下会遭受显著的精度下降，或者在部署时需要额外的计算开销，使其难以应用于工业界的大规模应用。在本文中，我们提出了decoupleQ，在极低比特下显著提升了模型精度。decoupleQ摒弃了传统的启发式量化范式，将模型参数解耦为整数和浮点数两部分，从而将量化问题转化为带约束的传统数学优化问题，并通过现成的优化方法交替求解。通过decoupleQ进行的量化是线性和均匀的，相比非均匀量化对硬件更友好，并且使得该思想可以迁移到高比特量化以增强其鲁棒性。我们的方法在字节跳动的大型语音模型的2比特量化上取得了接近fp16/bf16的在线精度。代码可在https://github.com/bytedance/decoupleQ获取。
英文摘要：
Quantization emerges as one of the most promising compression technologies for deploying efficient large models for various real time application in recent years. Considering that the storage and IO of weights take up the vast majority of the overhead inside a large model, weight only quantization can lead to large gains. However, existing quantization schemes suffer from significant accuracy degradation at very low bits, or require some additional computational overhead when deployed, making it difficult to be applied to large-scale applications in industry. In this paper, we propose decoupleQ, achieving a substantial increase in model accuracy, especially at very low bits. decoupleQ abandons the traditional heuristic quantization paradigm and decouples the model parameters into integer and floating-point parts, thus transforming the quantization problem into a traditional mathematical optimization problem with constraints, which is then solved alternatively by off-the-shelf optimization methods. Quantization via decoupleQ is linear and uniform, making it hardware-friendlier than non-uniform counterpart, and enabling the idea to be migrated to high-bit quantization to enhance its robustness. Our method has achieved well on-line accuracy near fp16/bf16 on the 2-bit quantization of large speech models in ByteDance. The code is available at https://github.com/bytedance/decoupleQ
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
