arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2511.04073cs.LGcs.DBcs.IR
收藏

面向多过滤器最近邻搜索的过滤器感知距离度量学习

Learning Filter-Aware Distance Metrics for Nearest Neighbor Search with Multiple Filters

Microsoft Research India(微软印度研究院);Microsoft Corporation(微软公司)

Ananya Sutradhar, Suryansh Gupta, Ravishankar Krishnaswamy, Haiyang Xu, Aseem Rastogi, Gopal Srinivasa

AI总结:

本文针对多过滤器近似最近邻搜索提出从数据中学习向量距离与过滤器匹配最优权衡的方法,通过约束线性优化得到自适应权重并用于搜索和索引构建,实验表明准确率比固定惩罚方法提高5-10%。

摘要:

过滤式近似最近邻(ANN)搜索从数据集中检索与查询向量最接近的向量,并要求查询指定的离散标签集合 $S$ 必须包含在每个被检索向量的标签中。现有的基于图的方法通常通过分配固定惩罚或基于过滤器满足情况对节点进行优先级排序来引入过滤器感知。然而,由于这些方法使用固定的、与数据无关的惩罚,它们在标签和向量分布多样的数据集上往往难以泛化。在本工作中,我们提出了一种有原则的替代方案,直接从数据中学习向量距离与过滤器匹配之间的最优权衡,而不是依赖固定惩罚。我们将该问题形式化为一个带约束的线性优化问题,推导出能够更好地反映底层过滤器分布的权重,从而更有效地解决过滤式 ANN 搜索问题。这些学习到的权重同时指导搜索过程和索引构建,使图结构能够更有效地捕捉底层过滤器分布和过滤器语义。我们的实验表明,使距离函数适应数据相比固定惩罚方法能够显著将准确率提高 5-10%,为过滤式 ANN 搜索问题提供了一个更灵活、更具泛化能力的框架。

英文摘要:

Filtered Approximate Nearest Neighbor (ANN) search retrieves the closest vectors for a query vector from a dataset. It enforces that a specified set of discrete labels $S$ for the query must be included in the labels of each retrieved vector. Existing graph-based methods typically incorporate filter awareness by assigning fixed penalties or prioritizing nodes based on filter satisfaction. However, since these methods use fixed, data in- dependent penalties, they often fail to generalize across datasets with diverse label and vector distributions. In this work, we propose a principled alternative that learns the optimal trade-off between vector distance and filter match directly from the data, rather than relying on fixed penalties. We formulate this as a constrained linear optimization problem, deriving weights that better reflect the underlying filter distribution and more effectively address the filtered ANN search problem. These learned weights guide both the search process and index construction, leading to graph structures that more effectively capture the underlying filter distribution and filter semantics. Our experiments demonstrate that adapting the distance function to the data significantly im- proves accuracy by 5-10% over fixed-penalty methods, providing a more flexible and generalizable framework for the filtered ANN search problem.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑