弥合稠密与稀疏最大内积搜索之间的鸿沟
Bridging Dense and Sparse Maximum Inner Product Search
AI总结:
本研究打破稠密与稀疏最大内积搜索(MIPS)的研究分化,验证了基于IVF的检索方法可高效解决稀疏MIPS问题,还提出了新型倒排索引结构与稠稀子空间统一MIPS框架。
摘要:
数十年来,针对稠密向量与稀疏向量的最大内积搜索(Maximum Inner Product Search,MIPS)在分化的研究文献中各自独立发展;其中稀疏向量MIPS在信息检索领域更以top-$k$检索为人熟知。尽管二者本质上是同一数学问题的不同表现形式,但由于稀疏与稠密向量服务于不同的最终目标,这种二元分化一直存在。在本研究中,我们探讨了稠密向量算法能否有效应用于稀疏向量,尤其是那些不满足top-$k$检索方法底层假设的稀疏向量场景。我们研究了基于IVF(倒排文件)的检索方法,该方法将向量划分到不同簇中,检索时仅搜索部分簇。我们对稀疏向量的降维方法开展了全面分析,并检验了标准KMeans与球面KMeans的聚类划分效果。实验结果表明,IVF是解决稀疏MIPS问题的高效方案。作为研究副产物,我们发现了两个研究方向并验证了其潜力:其一,我们将IVF范式视为一种动态剪枝技术,并基于该思路提出了一种面向通用稀疏向量近似MIPS的倒排索引新型组织结构;其二,我们提出了一套适用于同时包含稠密与稀疏子空间向量的MIPS统一框架,并证明了其对查询分布的鲁棒性。
英文摘要:
Maximum inner product search (MIPS) over dense and sparse vectors have progressed independently in a bifurcated literature for decades; the latter is better known as top-$k$ retrieval in Information Retrieval. This duality exists because sparse and dense vectors serve different end goals. That is despite the fact that they are manifestations of the same mathematical problem. In this work, we ask if algorithms for dense vectors could be applied effectively to sparse vectors, particularly those that violate the assumptions underlying top-$k$ retrieval methods. We study IVF-based retrieval where vectors are partitioned into clusters and only a fraction of clusters are searched during retrieval. We conduct a comprehensive analysis of dimensionality reduction for sparse vectors, and examine standard and spherical KMeans for partitioning. Our experiments demonstrate that IVF serves as an efficient solution for sparse MIPS. As byproducts, we identify two research opportunities and demonstrate their potential. First, we cast the IVF paradigm as a dynamic pruning technique and turn that insight into a novel organization of the inverted index for approximate MIPS for general sparse vectors. Second, we offer a unified regime for MIPS over vectors that have dense and sparse subspaces, and show its robustness to query distributions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。