QuIP#：结合Hadamard非相干性与格码本实现更优的LLM量化
QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks
AI总结：
本文提出QuIP#，一种仅权重的PTQ方法，通过随机化Hadamard变换、基于E8格的向量量化和微调，在每权重$\le$4比特的极端压缩下实现最先进性能，支持快速推理。
摘要：
训练后量化（PTQ）通过将LLM的权重量化为低精度来减少其内存占用。在本研究中，我们提出了QuIP#，这是一种仅权重的PTQ方法，它使用三种新技术在极端压缩机制（每权重$\le$ 4比特）下取得了最先进的结果。首先，QuIP#通过使用随机化Hadamard变换改进了QuIP（Chee等人，2023）的非相干性处理，这速度更快且具有更好的理论性质。其次，QuIP#使用向量量化来利用非相干权重所具备的球状亚高斯分布：具体而言，我们引入了一组基于高度对称$E_8$格的硬件高效码本，实现了最优的8维单位球填充。第三，QuIP#使用微调来提高对原始模型的保真度。我们的实验表明，QuIP#优于现有的PTQ方法，实现了PTQ缩放中的新行为，并支持快速推理。我们的代码可在https://github.com/Cornell-RelaxML/quip-sharp找到。
英文摘要：
Post-training quantization (PTQ) reduces the memory footprint of LLMs by quantizing their weights to low-precision. In this work, we introduce QuIP#, a weight-only PTQ method that achieves state-of-the-art results in extreme compression regimes ($\le$ 4 bits per weight) using three novel techniques. First, QuIP# improves QuIP's (Chee et al., 2023) incoherence processing by using the randomized Hadamard transform, which is faster and has better theoretical properties. Second, QuIP# uses vector quantization to take advantage of the ball-shaped sub-Gaussian distribution that incoherent weights possess: specifically, we introduce a set of hardware-efficient codebooks based on the highly symmetric $E_8$ lattice, which achieves the optimal 8-dimension unit ball packing. Third, QuIP# uses fine-tuning to improve fidelity to the original model. Our experiments show that QuIP# outperforms existing PTQ methods, enables new behaviors in PTQ scaling, and supports fast inference. Our code can be found at https://github.com/Cornell-RelaxML/quip-sharp.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
