Faster Maximum Inner Product Search in High Dimensions
Stanford University(斯坦福大学);Oxford University(牛津大学);University College London(伦敦大学学院);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Harvard T.H. Chan School of Public Health(哈佛陈曾熙公共卫生学院)
Mo Tiwari, Ryan Kang, Je-Yong Lee, Donghyun Lee, Chris Piech, Sebastian Thrun, Ilan Shomorony, Martin Jinye Zhang
Maximum Inner Product Search (MIPS) is a ubiquitous task in machine learning applications such as recommendation systems. Given a query vector and $n$ atom vectors in $d$-dimensional space, the goal of MIPS is to find the atom that has the highest inner product with the query vector. Existing MIPS algorithms scale at least as $O(\sqrt{d})$, which becomes computationally prohibitive in high-dimensional settings. In this work, we present BanditMIPS, a novel randomized MIPS algorithm whose complexity is independent of $d$. BanditMIPS estimates the inner product for each atom by subsampling coordinates and adaptively evaluates more coordinates for more promising atoms. The specific adaptive sampling strategy is motivated by multi-armed bandits. We provide theoretical guarantees that BanditMIPS returns the correct answer with high probability, while improving the complexity in $d$ from $O(\sqrt{d})$ to $O(1)$. We also perform experiments on four synthetic and real-world datasets and demonstrate that BanditMIPS outperforms prior state-of-the-art algorithms. For example, in the Movie Lens dataset ($n$=4,000, $d$=6,000), BanditMIPS is 20$\times$ faster than the next best algorithm while returning the same answer. BanditMIPS requires no preprocessing of the data and includes a hyperparameter that practitioners may use to trade off accuracy and runtime. We also propose a variant of our algorithm, named BanditMIPS-$α$, which achieves further speedups by employing non-uniform sampling across coordinates. Finally, we demonstrate how known preprocessing techniques can be used to further accelerate BanditMIPS, and discuss applications to Matching Pursuit and Fourier analysis.
SAH: Shifting-aware Asymmetric Hashing for Reverse $k$-Maximum Inner Product Search
School of Computing, National University of Singapore(新加坡国立大学计算机学院);School of Data Science and Engineering, East China Normal University(华东师范大学数据科学与工程学院)
Point Cloud Synthesis Using Inner Product Transforms
AIDOS Lab, University of Fribourg(AIDOS实验室,弗里堡大学);Institute of AI for Health, Helmholtz Munich(健康人工智能研究所,海德堡慕尼黑);Technical University of Munich(慕尼黑技术大学)