Multi-vector retrieval represents each query and document as a set of embeddings and scores relevance via fine-grained set-to-set interactions such as MaxSim (Chamfer similarity). This expressiveness improves retrieval quality over single-vector representations, but makes search substantially more expensive. Existing systems address this challenge primarily through proxy representations or filter-and-rerank pipelines that reduce candidate generation to single-vector search. We study the complementary direction of building a native IVF-style index directly over multi-vector data.
We introduce MV-IVF, an inverted-file index for multi-vector retrieval built on MV-Lloyd, a new Lloyd-style algorithm for clustering point clouds under a Chamfer objective. Our key observation is that once point clouds are assigned to clusters, the center-update step reduces to standard single-vector clustering over the flattened embeddings in each cluster. This enables efficient construction of a hierarchical IVF tree with center point clouds at internal nodes.
At query time, MV-IVF performs best-first beam search over the tree to identify promising leaf clusters, then scores the documents in those leaves. We further accelerate search using low-bit quantization, query compression, and cache-efficient batched search. We implement MV-IVF in MVSIC, an optimized C++ library and unified evaluation framework for multi-vector retrieval that includes several major approaches. Across BEIR and LoTTE text retrieval benchmarks, MV-IVF achieves strong latency--recall tradeoffs, the best batched throughput frontiers, and favorable build-time and index-size profiles. At $90\%$ Recall-$10$@$10$, MV-IVF achieves up to $8.03\times$ lower single-query latency and up to $173.5\times$ higher QPS than the baselines, demonstrating that native IVF-style indexing is practical and effective for scalable multi-vector retrieval.
Distill-VQ: Learning Retrieval Oriented Vector Quantization By Distilling Knowledge from Dense Embeddings
Beijing University of Posts and Telecommunications(北京邮电大学);Microsoft Research Asia(微软亚洲研究院);Microsoft Search Technology Center Asia(微软亚洲搜索技术中心);University of Science and Technology of China(中国科学技术大学)
Modelling-based experiment retrieval: A case study with gene expression clustering
Helsinki Institute for Information Technology HIIT(赫尔辛基信息技术研究所(HIIT));Aalto University(阿尔托大学);European Molecular Biology Laboratory(欧洲分子生物学实验室);European Bioinformatics Institute(欧洲生物信息研究所)
Queries Are Not Alone: Clustering Text Embeddings for Video Search
National Engineering Research Center for Software Engineering, Peking University(软件工程国家工程研究中心,北京大学);Peking University(北京大学);City University of Hong Kong(香港城市大学)
CRISP -- Clustering-Based Redundancy-Reduced Instance Sampling for Pathology Case Representation and Retrieval
Kimia Lab, Department of Artificial Intelligence & Informatics, Mayo Clinic, Rochester, MN, USA(Kimia实验室,人工智能与信息学系,梅奥诊所,罗切斯特,明尼苏达州,美国);DICE Lab, Department of Electrical and Computer Engineering, University of Illinois Chicago, IL, USA(DICE实验室,电气与计算机工程系,伊利诺伊大学芝加哥分校,伊利诺伊州,美国);MD Kimia Lab, Department of Artificial Intelligence & Informatics, Mayo Clinic, Rochester, MN, USA(MD Kimia实验室,人工智能与信息学系,梅奥诊所,罗切斯特,明尼苏达州,美国);PhD Kimia Lab, Department of Artificial Intelligence & Informatics, Mayo Clinic, Rochester, MN, USA(PhD Kimia实验室,人工智能与信息学系,梅奥诊所,罗切斯特,明尼苏达州,美国);Division of Computational Pathology and Informatics, Mayo Clinic, Rochester, MN, USA(计算病理学与信息学部,梅奥诊所,罗切斯特,明尼苏达州,美国);Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA(实验室医学与病理学系,梅奥诊所,罗切斯特,明尼苏达州,美国);Department of Breast and Melanoma Surgical Oncology, Comprehensive Cancer Center, Mayo Clinic, Rochester, MN, USA(乳腺和黑色素瘤外科肿瘤学系,综合癌症中心,梅奥诊所,罗切斯特,明尼苏达州,美国);Department of Oncology, Comprehensive Cancer Center, Mayo Clinic, Rochester, MN, USA(肿瘤学系,综合癌症中心,梅奥诊所,罗切斯特,明尼苏达州,美国);PhD H.R. Tizhoosh
Visual Late Chunking: An Empirical Study of Contextual Chunking for Efficient Visual Document Retrieval
Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Alibaba Cloud Computing(阿里云);Hong Kong University of Science and Technology(香港科技大学)