arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1903.08690cs.LGstat.ML
收藏

混合空间中的有效内积逼近

Efficient Inner Product Approximation in Hybrid Spaces

Google Research(谷歌研究院)

Xiang Wu, Ruiqi Guo, David Simcha, Dave Dopson, Sanjiv Kumar

摘要:

数据采掘和机器学习的许多新兴用例都在大型数据集上运行,这些数据集包含来自异构源的数据,特别是包含稀疏和密集组件的数据。例如,密集深度神经网络嵌入向量通常与稀疏文本特征结合使用,以提供文档的高维混合表示。在这种混合空间中进行有效搜索是非常具有挑战性的,因为对于稀疏向量表现良好的技术与对于密集向量表现良好的技术几乎没有重叠。流行的技术,如局部敏感散列(LSH)及其数据依赖的变体也没有在高维混合空间中提供良好的准确性。尽管混合场景变得越来越普遍,但目前文献中还没有既快速又准确的有效技术。在本文中,我们提出了一种技术,近似的混合向量的内积计算,从而大大加快搜索,同时保持高精度。我们还提出了有效的数据结构,利用现代计算机体系结构,导致数量级更快的搜索比现有的基线。所提出的方法的性能在几个数据集上得到了证明,包括一个非常大规模的工业数据集,在十亿维空间中包含十亿个向量,与竞争基线相比,实现了超过10倍的加速和更高的准确性。

英文摘要:

Many emerging use cases of data mining and machine learning operate on large datasets with data from heterogeneous sources, specifically with both sparse and dense components. For example, dense deep neural network embedding vectors are often used in conjunction with sparse textual features to provide high dimensional hybrid representation of documents. Efficient search in such hybrid spaces is very challenging as the techniques that perform well for sparse vectors have little overlap with those that work well for dense vectors. Popular techniques like Locality Sensitive Hashing (LSH) and its data-dependent variants also do not give good accuracy in high dimensional hybrid spaces. Even though hybrid scenarios are becoming more prevalent, currently there exist no efficient techniques in literature that are both fast and accurate. In this paper, we propose a technique that approximates the inner product computation in hybrid vectors, leading to substantial speedup in search while maintaining high accuracy. We also propose efficient data structures that exploit modern computer architectures, resulting in orders of magnitude faster search than the existing baselines. The performance of the proposed method is demonstrated on several datasets including a very large scale industrial dataset containing one billion vectors in a billion dimensional space, achieving over 10x speedup and higher accuracy against competitive baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑