多索引合并用于近似最近邻搜索
Reverse Neighbor Sliding and Order Selection for Efficient Multi-Proximity Graph Merging
AI总结：
本文提出了一种反向邻居滑动合并方法，通过优化多索引合并顺序，显著提升高维空间中近似最近邻搜索的效率。
摘要：
高维空间中的近似k最近邻（AKNN）搜索是向量数据库中的基础问题，具有广泛的应用。在众多AKNN索引中，基于接近图的索引在各种基准测试中实现了最先进的搜索效率。然而，它们对高维向量的大量距离计算导致了构建缓慢和显著的内存开销。有限的内存容量通常会阻止在处理大规模数据集时构建整个索引。常见的做法是分别构建多个子索引。然而，直接在这些分离的索引上进行搜索会严重损害搜索效率，因为查询无法利用跨图连接。因此，高效的图索引合并对于多索引搜索至关重要。在本文中，我们专注于高效的双索引合并以及多个索引的合并顺序。为此，我们提出了一种反向邻居滑动合并（RNSM），利用结构信息来提高合并效率。我们进一步研究了合并顺序选择（MOS），以通过消除冗余的合并操作来减少合并成本。实验表明，我们的方法在现有索引合并方法上实现了高达5.48倍的速度提升，并在索引重建上实现了9.92倍的速度提升，同时保持预期的优越搜索性能。此外，我们的方法能够高效地扩展到1亿个向量，使用50个分区，保持一致的速度提升。
英文摘要：
Approximate k Nearest Neighbor (AKNN) search in high-dimensional space is a foundational problem in vector databases with widespread applications. Among the numerous AKNN indexes, Proximity Graph-based indexes achieve state-of-the-art search efficiency across various benchmarks. In many real-world scenarios, datasets are maintained as multiple segment-level graph indexes to support continuous writes and segment management. However, these fragmented indexes complicate maintenance and degrade search efficiency, making fast graph index merging essential. In this paper, we focus on the efficient merging of multiple existing graph indexes into a single one. To achieve this, we propose a Reverse Neighbor Sliding Merge (RNSM) that exploits structural information to boost merging efficiency. We further propose Merge Order Selection (MOS) to minimize total merge cost across multiple indexes by eliminating redundant operations. Experiments show that our approach yields up to a 3.86x speedup over existing index merge methods and a 9.92x speedup over index reconstruction, while maintaining comparable search performance. Moreover, our method scales to merging up to 50 sub-indexes on datasets of 100 million vectors, maintaining consistent speedups.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
