用于近似最近邻搜索的向量数据库系统的综合实证评估：性能、质量与资源权衡
A Comprehensive Empirical Evaluation of Vector Database Systems for Approximate Nearest Neighbor Search: Performance, Quality, and Resource Trade-offs
浏览论文内容
中文总结 AI 辅助
本文对7款主流向量数据库系统开展综合实证评估，测量多维度指标，推导系统选择指南并发布开源基准框架，为AI应用选型提供依据。
中文摘要 AI 辅助
向量数据库已成为现代人工智能应用的关键基础设施，尤其适用于检索增强生成（RAG）、语义搜索及推荐系统。尽管其重要性日益提升，但仍缺乏全面且可复现的基准测试，以联合评估检索质量、查询延迟、吞吐量及资源利用率。本文对7款主流向量数据库系统开展系统性实证评估：FAISS、Qdrant、Milvus、Weaviate、Chroma、pgvector与LanceDB。评估方法涵盖6种不同数据集，从经典计算机视觉描述符（SIFT、GIST）到基于Transformer的文本嵌入（MS MARCO、GloVe），包含超400万个维度介于96至960的向量。我们测量了15项指标，涵盖检索质量（Recall@K、Precision@K、MRR、NDCG@K、Hit Rate@K）、查询性能（延迟百分位、QPS、冷启动延迟）及资源消耗（索引构建时间、内存、存储）。在SIFT1M数据集上，FAISS实现了最高的单节点吞吐量（866 QPS），但缺少数据库操作功能；Weaviate提供了最佳的开箱即用召回率（>99%）；Qdrant在全功能数据库中延迟最优（中位数4.55ms）；LanceDB则以牺牲检索质量为代价，换取大幅更快的索引构建速度。我们为从业者推导了系统选择指南，并将基准测试框架作为开源软件发布。
英文摘要
Vector databases have emerged as critical infrastructure for modern artificial intelligence applications, particularly retrieval-augmented generation (RAG), semantic search, and recommendation systems. Despite their growing importance, there remains a significant gap in comprehensive, reproducible benchmarks that jointly evaluate retrieval quality, query latency, throughput, and resource utilization. We present a systematic empirical evaluation of seven prominent vector database systems: FAISS, Qdrant, Milvus, Weaviate, Chroma, pgvector, and LanceDB. Our methodology spans six diverse datasets, from classical computer-vision descriptors (SIFT, GIST) to transformer-based text embeddings (MS MARCO, GloVe), encompassing over 4 million vectors at dimensionalities from 96 to 960. We measure 15 metrics spanning retrieval quality (Recall@K, Precision@K, MRR, NDCG@K, Hit Rate@K), query performance (latency percentiles, QPS, cold-start latency), and resource consumption (index build time, memory, storage). On SIFT1M, FAISS achieves the highest single-node throughput (866 QPS) but lacks database operational features; Weaviate provides the best out-of-the-box recall (> 99%); Qdrant offers the best latency among full databases (4.55~ms median); and LanceDB trades retrieval quality for substantially faster index construction. We derive system-selection guidelines for practitioners and release our benchmarking framework as open-source software.
