ACORN:面向向量嵌入与结构化数据的高性能谓词无关搜索
ACORN: Performant and Predicate-Agnostic Search Over Vector Embeddings and Structured Data
AI总结:
针对混合搜索现有方法性能差、谓词支持有限的问题,提出基于HNSW的谓词无关混合搜索方法ACORN,通过谓词子图遍历实现高效搜索,在多类数据集上吞吐量较现有方法提升2-1000倍。
摘要:
越来越多的应用开始利用多模态数据,需要同时对向量数据(如图像、文本和视频嵌入)以及结构化数据(如属性和关键词)进行联合搜索。目前针对这种混合搜索场景提出的方法要么性能不佳,要么仅支持极其有限的搜索谓词集合(例如仅支持少量相等谓词),导致其在许多应用中不具备实用性。为解决这一问题,我们提出了ACORN,一种高性能且谓词无关的混合搜索方法。ACORN基于层次化可导航小世界(Hierarchical Navigable Small Worlds, HNSW)——一种当前最先进的基于图的近似最近邻索引——构建,可通过扩展现有HNSW库实现高效部署。ACORN引入了谓词子图遍历的思想,用以模拟一种理论上理想但实际难以实现的混合搜索策略。ACORN的谓词无关构建算法旨在支撑这一高效搜索策略,同时支持多种谓词集合与查询语义。我们在两类数据集上对ACORN进行了系统评估:一类是已有基准数据集,包含简单、低基数的谓词集合;另一类是现有方法无法支持的复杂多模态数据集。实验结果表明,ACORN在所有数据集上均达到了当前最优性能,在固定召回率下,其吞吐量比现有方法高出2至1000倍。
英文摘要:
Applications increasingly leverage mixed-modality data, and must jointly search over vector data, such as embedded images, text and video, as well as structured data, such as attributes and keywords. Proposed methods for this hybrid search setting either suffer from poor performance or support a severely restricted set of search predicates (e.g., only small sets of equality predicates), making them impractical for many applications. To address this, we present ACORN, an approach for performant and predicate-agnostic hybrid search. ACORN builds on Hierarchical Navigable Small Worlds (HNSW), a state-of-the-art graph-based approximate nearest neighbor index, and can be implemented efficiently by extending existing HNSW libraries. ACORN introduces the idea of predicate subgraph traversal to emulate a theoretically ideal, but impractical, hybrid search strategy. ACORN's predicate-agnostic construction algorithm is designed to enable this effective search strategy, while supporting a wide array of predicate sets and query semantics. We systematically evaluate ACORN on both prior benchmark datasets, with simple, low-cardinality predicate sets, and complex multi-modal datasets not supported by prior methods. We show that ACORN achieves state-of-the-art performance on all datasets, outperforming prior methods with 2-1,000x higher throughput at a fixed recall.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。