arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1810.09104cs.LGstat.ML
收藏

范数-值域划分:基于LSH的最大内积搜索(MIPS)的通用催化剂

Norm-Range Partition: A Universal Catalyst for LSH based Maximum Inner Product Search (MIPS)

The Chinese University of Hong Kong(香港中文大学)

Xiao Yan, Xinyan Dai, Jie Liu, Kaiwen Zhou, James Cheng

摘要:

最近,局部敏感哈希(LSH)被证明是有效的MIPS和几个算法,包括$L_2$ -ALSH,Sign-ALSH和Simple-LSH已被提出。本文介绍了范数-范围划分技术,该技术将原始数据集划分为包含具有相似2-范数的项的子数据集,并为每个子数据集独立地建立哈希索引。我们证明了范数-范围划分降低了现有的基于LSH的MIPS算法在温和的条件下的查询处理复杂度。提高性能的关键是范数范围划分允许使用较小的归一化因子的大多数子数据集。为了有效的查询处理,我们还制定了一个统一的框架来对不同子数据集的哈希索引中的桶进行排名。在真实数据集上的实验表明,在达到相同召回率的情况下,范数-范围划分显著减少了基于LSH的MIPS算法的探测次数。

英文摘要:

Recently, locality sensitive hashing (LSH) was shown to be effective for MIPS and several algorithms including $L_2$-ALSH, Sign-ALSH and Simple-LSH have been proposed. In this paper, we introduce the norm-range partition technique, which partitions the original dataset into sub-datasets containing items with similar 2-norms and builds hash index independently for each sub-dataset. We prove that norm-range partition reduces the query processing complexity for all existing LSH based MIPS algorithms under mild conditions. The key to performance improvement is that norm-range partition allows to use smaller normalization factor most sub-datasets. For efficient query processing, we also formulate a unified framework to rank the buckets from the hash indexes of different sub-datasets. Experiments on real datasets show that norm-range partition significantly reduces the number of probed for LSH based MIPS algorithms when achieving the same recall.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑