VADER:具有声明性召回率的过滤向量搜索
VADER: Filtered Vector Search with Declarative Recall
- Université Paris Cité(巴黎西岱大学)
- Brown University(布朗大学)
- Google(谷歌)
- ETH Zürich(苏黎世联邦理工学院)
机构由 AI 辅助整理,请以论文原文为准。
中文总结 AI 辅助
VADER通过声明性召回率消除超参数调整,利用过滤感知预测器实现提前终止,在过滤向量搜索中达到近最优性能,比基线快53%、质量高28%。
中文摘要 AI 辅助
近似过滤向量搜索(FVS)是许多将结构化数据与向量嵌入相结合的数据管理任务中的核心操作,由于过滤谓词的特征,其复杂度有所增加。每个谓词由选择性(即满足谓词的向量所占比例)和相关性(即过滤器与向量空间之间的关系)定义,即使对于相同的查询向量,这些因素也会显著影响搜索难度。这对旨在将向量搜索与结构化数据集成的用户构成了关键挑战,因为高效执行通常需要对算法参数进行大量手动调整。在本文中,我们提出了VADER,这是第一种通过为近似过滤向量搜索引入声明性召回率来消除超参数调整的方法。通过声明性召回率,用户指定期望的召回率目标,VADER执行FVS查询以满足该目标,无需手动配置。VADER通过采用一种过滤感知的召回率预测器来实现这一点,该预测器无需显式调整即可在不同选择性和相关性之间进行泛化,并在预测召回率达到用户定义的目标时提前终止。通过广泛的实验评估,我们表明VADER实现了接近最优的提前终止,同时与最佳基线相比,提供了高达53%的显著加速和28%的结果质量提升。
英文摘要
Approximate filtered vector search (FVS), a core operation in many data management tasks that combine structured data with vector embeddings, exhibits increased complexity due to the characteristics of filtering predicates. Each predicate is defined by selectivity (i.e., the fraction of vectors that satisfy the predicate) and correlation (i.e., the relationship between the filter and the vector space), which can significantly affect search difficulty even for the same query vector. This poses a key challenge for users aiming to integrate vector search with structured data, as efficient execution often requires extensive manual tuning of algorithm parameters. In this paper, we present VADER, the first approach that eliminates hyperparameter tuning by introducing declarative recall for approximate filtered vector search. With declarative recall, users specify a desired recall target, and VADER executes FVS queries to meet this target without requiring manual configuration. VADER achieves this by employing a filter-aware recall predictor that generalizes across varying selectivities and correlations without explicit tuning, and by performing early termination once the predicted recall reaches the user-defined target. Through extensive experimental evaluation, we show that VADER achieves near-optimal early termination, while providing significant speedups of up to 53% faster and improved result quality of 28% compared to the best-performing baseline.