arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2212.07551cs.LGcs.AI
收藏

高维空间中更快的最大内积搜索

Faster Maximum Inner Product Search in High Dimensions

Stanford University(斯坦福大学);Oxford University(牛津大学);University College London(伦敦大学学院);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Harvard T.H. Chan School of Public Health(哈佛陈曾熙公共卫生学院)

Mo Tiwari, Ryan Kang, Je-Yong Lee, Donghyun Lee, Chris Piech, Sebastian Thrun, Ilan Shomorony, Martin Jinye Zhang

AI总结:

针对高维最大内积搜索(MIPS)计算瓶颈,提出复杂度与维度无关的随机算法 BanditMIPS,通过多臂老虎机启发的自适应坐标采样策略将复杂度降至 O(1),在保持高精度的同时显著提升速度。

摘要:

最大内积搜索(MIPS)是推荐系统等机器学习应用中普遍存在的任务。给定一个查询向量和 $d$ 维空间中的 $n$ 个原子向量,MIPS 的目标是找到与查询向量内积最高的原子。现有的 MIPS 算法的复杂度至少为 $O(\sqrt{d})$,这在高维设置中会产生计算上的 prohibitive。在这项工作中,我们提出了 BanditMIPS,一种新颖的随机化 MIPS 算法,其复杂度与 $d$ 无关。BanditMIPS 通过对坐标进行子采样来估计每个原子的内积,并对更有希望的原子自适应地评估更多坐标。这种特定的自适应采样策略是由多臂老虎机(multi-armed bandits)启发的。我们提供了理论保证,证明 BanditMIPS 能以高概率返回正确答案,同时将 $d$ 中的复杂度从 $O(\sqrt{d})$ 提高到 $O(1)$。我们还在四个合成和真实世界数据集上进行了实验,证明 BanditMIPS 优于先前的最先进算法。例如,在 Movie Lens 数据集($n$=4,000,$d$=6,000)中,BanditMIPS 比次优算法快 20 倍,同时返回相同的答案。BanditMIPS 不需要对数据进行预处理,并包含一个超参数,从业者可以使用该超参数来权衡准确性和运行时间。我们还提出了算法的一个变体,名为 BanditMIPS-$α$,它通过在坐标间采用非均匀采样来实现进一步的加速。最后,我们展示了如何使用已知的预处理技术来进一步加速 BanditMIPS,并讨论了其在 Matching Pursuit 和 Fourier 分析中的应用。

英文摘要:

Maximum Inner Product Search (MIPS) is a ubiquitous task in machine learning applications such as recommendation systems. Given a query vector and $n$ atom vectors in $d$-dimensional space, the goal of MIPS is to find the atom that has the highest inner product with the query vector. Existing MIPS algorithms scale at least as $O(\sqrt{d})$, which becomes computationally prohibitive in high-dimensional settings. In this work, we present BanditMIPS, a novel randomized MIPS algorithm whose complexity is independent of $d$. BanditMIPS estimates the inner product for each atom by subsampling coordinates and adaptively evaluates more coordinates for more promising atoms. The specific adaptive sampling strategy is motivated by multi-armed bandits. We provide theoretical guarantees that BanditMIPS returns the correct answer with high probability, while improving the complexity in $d$ from $O(\sqrt{d})$ to $O(1)$. We also perform experiments on four synthetic and real-world datasets and demonstrate that BanditMIPS outperforms prior state-of-the-art algorithms. For example, in the Movie Lens dataset ($n$=4,000, $d$=6,000), BanditMIPS is 20$\times$ faster than the next best algorithm while returning the same answer. BanditMIPS requires no preprocessing of the data and includes a hyperparameter that practitioners may use to trade off accuracy and runtime. We also propose a variant of our algorithm, named BanditMIPS-$α$, which achieves further speedups by employing non-uniform sampling across coordinates. Finally, we demonstrate how known preprocessing techniques can be used to further accelerate BanditMIPS, and discuss applications to Matching Pursuit and Fourier analysis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑