arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.01065cs.LG
收藏

GSRQ: 面向亚1比特KV缓存的增益-形状残差量化

GSRQ: Gain-Shape Residual Quantization for Sub-1-bit KV Cache

Soosung Kim, Minjae Park, Eui-Young Chung, Jaeyong Chung

AI总结:

针对大语言模型KV缓存压缩中向量量化导致的方向保持问题,提出增益-形状K均值(GSKM)替代标准K均值,并构建增益-形状残差量化(GSRQ),在1比特下将LongBench平均准确率从11.34提升至33.54。

摘要:

随着上下文窗口的扩展,大语言模型(LLM)的部署越来越受到键值(KV)缓存内存线性增长的制约。向量量化(VQ),特别是残差量化(RQ),通过使用小码本逐步编码残差,是一种将KV缓存存储推向亚1比特领域的有前景的方法。然而,大多数VQ方法仍然依赖标准的$\ell_2$ $K$-均值作为核心码本学习原语。我们发现了该原语的一个微妙的高维问题:欧几里得质心平均可能导致质心收缩,这会削弱$\ell_2$失真中的角度对齐项,使得方向保持更加困难。为了解决这个问题,我们提出了增益-形状$K$-均值(GSKM),它是$K$-均值的即插即用替代品,在匹配甚至在某些情况下改善$\ell_2$失真的同时,提高了方向保真度。然后,我们通过将GSKM的加权扩展纳入RQ流水线,构建了增益-形状残差量化(GSRQ)。在LLaMA-3-8B上,GSRQ在不同比特率下显著优于强KV缓存量化基线。在1比特时,它将LongBench任务的平均准确率从11.34提升到33.54,相比VQLLM提高了22.20个百分点。

英文摘要:

The deployment of Large Language Models (LLMs) with extended context windows is increasingly constrained by the linear growth of Key-Value (KV) cache memory. Vector Quantization (VQ), particularly Residual Quantization (RQ), is a promising approach for pushing KV cache storage toward the sub-1-bit regime by progressively encoding residuals with small codebooks. However, most VQ methods still rely on standard $\ell_2$ $K$-means as the core codebook-learning primitive. We identify a subtle high-dimensional issue of this primitive: Euclidean centroid averaging can induce centroid shrinkage, which weakens the angular alignment term in the $\ell_2$ distortion and makes directional preservation harder. To address this issue, we propose Gain-Shape $K$-means (GSKM), a drop-in replacement for $K$-means that improves directional fidelity while matching, and in some regimes improving, $\ell_2$ distortion. We then build Gain-Shape Residual Quantization (GSRQ) by incorporating a weighted extension of GSKM into an RQ pipeline. On LLaMA-3-8B, GSRQ substantially improves over strong KV cache quantization baselines across bit rates. At 1-bit, it improves the average accuracy across LongBench tasks from 11.34 to 33.54, a gain of 22.20 percentage points over VQLLM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑