arXiv 2603.164352026-03-17cs.CL

VQKV：通过向量量化实现高保真和高比的缓存压缩

VQKV: High-Fidelity and High-Ratio Cache Compression via Vector-Quantization

LUMIA Lab（LUMIA实验室）；School of Artificial Intelligence（人工智能学院）；Shanghai Jiao Tong University（上海交通大学）；Shanghai Innovation Institute（上海创新研究院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Cambridge（剑桥大学）；National University of Singapore（新加坡国立大学）

Yixuan Wang, Qingyu Shi, Jiayu Zhou, Dianbo Liu, Ziwei He, Zhouhan Lin

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出VQKV方法，利用向量量化实现高压缩比和高重建保真度的KV缓存压缩，实现在有限资源环境下提升模型性能。

摘要：

随着大型语言模型（LLMs）上下文长度的增长，键值（KV）缓存的规模增大，限制了在资源受限环境中的部署。以往的无训练方法通常依赖于低秩近似或标量量化，难以同时实现高压缩比和高重建保真度。我们提出VQKV，一种新颖的无训练方法，引入向量量化（VQ）以获得高度压缩的KV表示，同时保持高模型保真度，允许仅用少量整数索引表示数千个浮点值。结果，VQKV在LLaMA3.1-8B上实现了82.8%的压缩比，保留了LongBench基准测试中98.6%的基线性能，并在相同内存足迹上实现了4.3倍更长的生成长度。

英文摘要：

The growing context length of Large Language Models (LLMs) enlarges the Key-Value (KV) cache, limiting deployment in resource-limited environments. Prior training-free approaches for KV cache compression typically rely on low-rank approximation or scalar quantization, which fail to simultaneously achieve high compression ratios and high reconstruction fidelity. We propose VQKV, a novel, training-free method introducing vector quantization (VQ) to obtain highly compressed KV representations while preserving high model fidelity, allowing for the representation of thousands of floating-point values with just a few integer indices. As a result, VQKV achieves an 82.8\% compression ratio on LLaMA3.1-8B while retaining 98.6\% of the baseline performance on LongBench and enabling 4.3x longer generation length on the same memory footprint.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。