QTIP：基于网格编码与非相干处理的量化方法
QTIP: Quantization with Trellises and Incoherence Processing
AI总结：
QTIP提出用网格编码量化替代向量量化，实现超高维LLM后训练量化，在量化质量和推理速度上均达到最先进水平。
摘要：
后训练量化（PTQ）通过将权重量化为低精度数据类型来减少大语言模型（LLM）的内存占用。由于LLM推理通常受内存带宽限制，PTQ方法能够提升推理吞吐量。近期最先进的PTQ方法采用向量量化（VQ）一次性量化多个权重，通过更好的形状塑造提高信息利用率。然而，VQ需要码本大小随维度呈指数增长，这限制了当前基于VQ的PTQ方法只能使用较低的VQ维度（≤8），进而限制了量化质量。在此，我们提出QTIP，该方法改用网格编码量化（TCQ）来实现超高维量化。TCQ使用有状态解码器，将码本大小与比特率和有效维度分离。QTIP引入了一系列从纯查找表到计算式无查找表的网格码，专为硬件高效的“位移”网格结构设计；这些码在量化质量和推理速度两方面均达到了最先进水平。
英文摘要：
Post-training quantization (PTQ) reduces the memory footprint of LLMs by quantizing weights to low-precision datatypes. Since LLM inference is usually memory-bound, PTQ methods can improve inference throughput. Recent state-of-the-art PTQ approaches use vector quantization (VQ) to quantize multiple weights at once, which improves information utilization through better shaping. However, VQ requires a codebook with size exponential in the dimension. This limits current VQ-based PTQ works to low VQ dimensions ($\le 8$) that in turn limit quantization quality. Here, we introduce QTIP, which instead uses trellis coded quantization (TCQ) to achieve ultra-high-dimensional quantization. TCQ uses a stateful decoder that separates the codebook size from the bitrate and effective dimension. QTIP introduces a spectrum of lookup-only to computed lookup-free trellis codes designed for a hardware-efficient "bitshift" trellis structure; these codes achieve state-of-the-art results in both quantization quality and inference speed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
