揭示隐藏的陷阱并导航下一代向量相似性搜索:从任务导向的角度
Reveal Hidden Pitfalls and Navigate Next Generation of Vector Similarity Search from Task-Centric Views
AI总结:
从任务导向角度提出Iceberg基准测试套件,揭示向量相似性搜索中信息损失漏斗的三大来源,并通过任务特定的元特征和决策树指导方法选择与调优。
摘要:
向量相似性搜索(VSS)在高维空间中的应用正迅速成为下一代数据库系统的核心功能,用于多种数据密集型服务——从大型语言模型中的嵌入查找,到语义信息检索和推荐引擎。然而,当前的基准测试主要基于召回率与延迟的权衡,仅通过距离度量定义的地面真实情况来评估VSS,忽略了检索质量最终如何影响下游任务。这种脱节可能会误导学术研究和工业实践。我们提出了Iceberg,一个全面的基准测试套件,用于对VSS方法进行端到端评估,以现实的应用场景为背景。从任务导向的角度来看,Iceberg揭示了信息损失漏斗,该漏斗识别了三个主要的端到端性能下降来源:(1)特征提取期间的嵌入损失;(2)度量误用,其中距离不佳地反映任务相关性;(3)数据分布敏感性,突显索引在偏斜和模态中的鲁棒性。为了更全面的评估,Iceberg跨越八个多样化的数据集,涵盖图像分类、人脸识别、文本检索和推荐系统等关键领域。每个数据集从100万到1亿向量不等,包含丰富的、任务特定的标签和评估指标,使检索算法能够在完整的应用流程中进行评估,而不是孤立地进行评估。Iceberg基准测试了13种最先进的VSS方法,并根据应用级别的指标重新排序,揭示了与传统仅基于召回率和延迟评估得出的排名有显著差异。基于这些见解,我们定义了一组任务导向的元特征,并推导出一个可解释的决策树,以指导从业者在特定工作负载中选择和调整VSS方法。
英文摘要:
Vector Similarity Search (VSS) in high-dimensional spaces is rapidly emerging as core functionality in next-generation database systems for numerous data-intensive services -- from embedding lookups in large language models (LLMs), to semantic information retrieval and recommendation engines. Current benchmarks, however, evaluate VSS primarily on the recall-latency trade-off against a ground truth defined solely by distance metrics, neglecting how retrieval quality ultimately impacts downstream tasks. This disconnect can mislead both academic research and industrial practice. We present Iceberg, a holistic benchmark suite for end-to-end evaluation of VSS methods in realistic application contexts. From a task-centric view, Iceberg uncovers the Information Loss Funnel, which identifies three principal sources of end-to-end performance degradation: (1) Embedding Loss during feature extraction; (2) Metric Misuse, where distances poorly reflect task relevance; (3) Data Distribution Sensitivity, highlighting index robustness across skews and modalities. For a more comprehensive assessment, Iceberg spans eight diverse datasets across key domains such as image classification, face recognition, text retrieval, and recommendation systems. Each dataset, ranging from 1M to 100M vectors, includes rich, task-specific labels and evaluation metrics, enabling assessment of retrieval algorithms within the full application pipeline rather than in isolation. Iceberg benchmarks 13 state-of-the-art VSS methods and re-ranks them based on application-level metrics, revealing substantial deviations from traditional rankings derived purely from recall-latency evaluations. Building on these insights, we define a set of task-centric meta-features and derive an interpretable decision tree to guide practitioners in selecting and tuning VSS methods for their specific workloads.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。