arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2501.03078cs.LG
收藏

Qinco2:利用改进的隐式神经码本进行向量压缩与搜索

Qinco2: Vector Compression and Search with Improved Implicit Neural Codebooks

Meta Fundamental AI Research(Meta基础AI研究院)

Théophane Vallaeys, Matthew Muckley, Jakob Verbeek, Matthijs Douze

AI总结:

本文提出QINCo2,通过码字预选、束搜索、码字对快速近似解码及优化训练与架构,提升向量压缩和十亿规模最近邻搜索性能。

摘要:

向量量化是压缩和大规模最近邻搜索的一项基础技术。对于高精度工作点,多码本量化将数据向量与来自多个码本中每一个码本的一个元素相关联。残差量化(RQ)就是一个例子,它迭代地量化先前步骤的残差误差。然而,RQ忽略了编码不同部分之间的依赖关系,这导致次优的率失真性能。QINCo最近通过使用神经网络基于先前步骤的向量重建来确定RQ中的量化码本,解决了这一低效问题。在本文中,我们介绍了QINCo2,它通过以下方面扩展并改进了QINCo:(i)使用码字预选和束搜索的改进向量编码,(ii)利用码字对为搜索建立准确候选短名单的快速近似解码器,以及(iii)优化的训练过程和网络架构。我们在四个数据集上进行实验,以评估QINCo2在向量压缩和十亿规模最近邻搜索方面的表现。我们在两种设置下都取得了出色结果:在BigANN上的16字节向量压缩中,将最先进的重建MSE改进了34%;在Deep1M上使用8字节编码时,将搜索准确率提高了24%。

英文摘要:

Vector quantization is a fundamental technique for compression and large-scale nearest neighbor search. For high-accuracy operating points, multi-codebook quantization associates data vectors with one element from each of multiple codebooks. An example is residual quantization (RQ), which iteratively quantizes the residual error of previous steps. Dependencies between the different parts of the code are, however, ignored in RQ, which leads to suboptimal rate-distortion performance. QINCo recently addressed this inefficiency by using a neural network to determine the quantization codebook in RQ based on the vector reconstruction from previous steps. In this paper we introduce QINCo2 which extends and improves QINCo with (i) improved vector encoding using codeword pre-selection and beam-search, (ii) a fast approximate decoder leveraging codeword pairs to establish accurate short-lists for search, and (iii) an optimized training procedure and network architecture. We conduct experiments on four datasets to evaluate QINCo2 for vector compression and billion-scale nearest neighbor search. We obtain outstanding results in both settings, improving the state-of-the-art reconstruction MSE by 34% for 16-byte vector compression on BigANN, and search accuracy by 24% with 8-byte encodings on Deep1M.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑