RACORN-1: 面向低选择性过滤向量搜索的自适应保召回加速方法
RACORN: Robust Predicate-Agnostic Filtered Vector Search under Extreme Selectivity
AI总结:
针对ACORN-1在低选择性下召回率崩溃的问题,提出RACORN-1,通过自适应搜索回退(ASF)利用过滤失败节点作为临时桥梁恢复连通性,并引入自适应精确回退(AEF)处理极低选择性场景,在保持高召回的同时实现9-26倍加速。
摘要:
过滤向量搜索(FVS)结合了向量嵌入相似性与结构化元数据谓词,已成为RAG和生产检索系统的核心需求。ACORN-1作为代表性的内过滤算法,重用现有的HNSW索引,在低选择性下显著降低延迟,但在选择性低于5%时出现连接不稳定性,低于1%时召回率崩溃。我们提出RACORN-1,这是ACORN-1的原位扩展,通过以下方法解决此崩溃:(i)自适应搜索回退(ASF)——将过滤失败的节点重新用作临时桥梁,绕过断裂路径;桥梁和两跳候选选择使用步长采样以实现空间多样性。虽然过滤优先的ACORN系列方法相对于距离优先的HNSW存在结构性的召回权衡,但RACORN-1通过ASF改进了权衡曲线,在显著降低延迟的同时最小化召回损失。在三个100万规模和一个4000万规模的数据集上,RACORN-1在最佳点(1%-0.3%选择性)相比HNSW实现了约9-26倍的延迟降低,并将ACORN-1的召回率从0.45-0.72(1%)和0.03-0.10(0.3%)分别恢复到0.70-0.96和0.77-0.98。对于线性扫描可能优于图搜索的极低选择性区域,我们将RACORN-1与(ii)自适应精确回退(AEF)结合在变体RACORN-1+中,在100万规模≤0.1%选择性下实现召回率1.00和20-75倍加速,在4000万规模0.01%选择性下实现13倍加速。在负相关性评估(K-means聚类)中,ACORN-1崩溃(召回率0.08-0.41),而RACORN-1保持召回率0.80-0.98,且延迟比HNSW低5-9倍。总之,RACORN-1和RACORN-1+构成了一个与ACORN-1兼容的机制,对极低选择性和对抗性查询-过滤相关性均具有鲁棒性。
英文摘要:
Filtered vector search returns the nearest neighbors among the vectors that pass a predicate. The predicate is often selective: most tag queries of the Big ANN filtered track select less than 1% of YFCC 10M. Predicate-agnostic methods search one HNSW graph built without metadata and so take any predicate, but below a few percent selectivity they break down. ACORN-1 draws candidates only from passing neighbors and its recall collapses below 1%. HNSW keeps searching but spends about ef/s expansion cycles at selectivity s. We present RACORN-1, an extension of ACORN-1 that keeps its graph and filter-first traversal and adds Adaptive Search Fallback. When the two-hop expansion of a node falls short of a target, failing nodes enter the candidate queue as bridges, under a per-expansion budget, but never enter the result queue. RACORN-1 serves the selectivities ACORN-1 cannot: from 1% down ACORN-1 no longer reaches recall 0.9 at any ef, while RACORN-1 reaches 0.95 from 3% down on every dataset we sweep. It also reaches 0.9 on every natural filter that YFCC and LAION ship and on every conjunction and caption predicate we run, where ACORN-1 misses more than half. With a deadline equal to the time HNSW needs to fill its result queue, RACORN-1 returns the higher recall on all eight datasets at four selectivities. On eight indexes of 1M to 100M vectors, at 1% selectivity and recall 0.9 it is 1.3-8.4x faster than HNSW, and it is 2.2-10.5x faster under adverse correlation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。