arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2107.05537cs.DB
收藏

PM-LSH:一个快速准确的内存框架,用于高维近似NN和最近对搜索

PM-LSH: a fast and accurate in-memory framework for high-dimensional approximate NN and closest pair search

Bolong Zheng, Xi Zhao, Lianggui Weng, Nguyen Quoc Viet Hung, Hang Liu, Christian S. Jensen

摘要:

最近邻(NN)搜索是固有的计算昂贵的高维空间中,由于维数灾难。作为一种著名的解决方案,局部敏感哈希(LSH)能够在次线性时间内以恒定概率回答c-近似神经网络(c-ANN)查询。现有的LSH方法主要集中在建立基于哈希桶的索引,以便可以快速检索候选点。然而,现有的粗粒度结构无法为候选点提供准确的距离估计,这在必须检查不必要的点时转化为额外的计算开销。这反过来又降低了查询处理的性能。相比之下,我们提出了一个快速准确的内存LSH框架,称为PM-LSH,旨在计算大规模,高维数据集上的c-ANN查询。首先,我们采用一个简单而有效的PM树索引的数据点。其次,我们开发了一个可调的置信区间,以实现准确的距离估计,并保证高质量的结果。第三,我们提出了一个有效的算法上的PM树,以提高计算c-ANN查询的性能。此外,我们扩展了PM-LSH,以支持高维空间中的最近对(CP)搜索。我们再次采用PM树来组织低维空间中的点,我们提出了一个分支定界算法与半径修剪技术,以提高计算c-近似最近对(c-ACP)查询的性能。大量的实验与真实世界的数据提供的证据表明,PM-LSH是能够优于现有的建议,在效率和准确性的NN和CP搜索。

英文摘要:

Nearest neighbor (NN) search is inherently computationally expensive in high-dimensional spaces due to the curse of dimensionality. As a well-known solution, locality-sensitive hashing (LSH) is able to answer c-approximate NN (c-ANN) queries in sublinear time with constant probability. Existing LSH methods focus mainly on building hash bucket-based indexing such that the candidate points can be retrieved quickly. However, existing coarse-grained structures fail to offer accurate distance estimation for candidate points, which translates into additional computational overhead when having to examine unnecessary points. This in turn reduces the performance of query processing. In contrast, we propose a fast and accurate in-memory LSH framework, called PM-LSH, that aims to compute c-ANN queries on large-scale, high-dimensional datasets. First, we adopt a simple yet effective PM-tree to index the data points. Second, we develop a tunable confidence interval to achieve accurate distance estimation and guarantee high result quality. Third, we propose an efficient algorithm on top of the PM-tree to improve the performance of computing c-ANN queries. In addition, we extend PM-LSH to support closest pair (CP) search in high-dimensional spaces. We again adopt the PM-tree to organize the points in a lowdimensional space, and we propose a branch and bound algorithm together with a radius pruning technique to improve the performance of computing c-approximate closest pair (c-ACP) queries. Extensive experiments with real-world data offer evidence that PM-LSH is capable of outperforming existing proposals with respect to both efficiency and accuracy for both NN and CP search.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑