多模态任意模态的多向量索引压缩
Multi-Vector Index Compression in Any Modality
AI总结：
本文提出注意力引导聚类方法，用于在任意模态中高效压缩多向量索引，提升检索性能。
摘要：
我们研究了在任何模态中进行后期交互的高效多向量检索。后期交互已成为文本、图像、视觉文档和视频信息检索中的主导范式，但其计算和存储成本随着文档长度线性增长，使图像、视频和音频丰富的语料库变得昂贵。为了解决这一限制，我们探索了压缩多向量文档表示的查询无关方法，在恒定向量预算下。我们介绍了四种索引压缩方法：序列缩放、内存令牌、层次池化和一种新的注意力引导聚类（AGC）。AGC使用注意力引导机制来识别文档中最具语义显著性的区域作为聚类质心，并对令牌聚合进行加权。在涵盖文本（BEIR）、视觉文档（ViDoRe）和视频（MSR-VTT、MultiVENT 2.0）的检索任务上评估这些方法，我们显示注意力引导聚类在其他参数化压缩方法（序列缩放和内存令牌）上表现一致，比非参数化层次聚类提供了更大的索引大小灵活性，并在与完整未压缩索引相比时实现了竞争或改进的性能。源代码可在：github.com/hanxiangqin/omni-col-press。
英文摘要：
We study efficient multi-vector retrieval for late interaction in any modality. Late interaction has emerged as a dominant paradigm for information retrieval in text, images, visual documents, and videos, but its computation and storage costs grow linearly with document length, making it costly for image-, video-, and audio-rich corpora. To address this limitation, we explore query-agnostic methods for compressing multi-vector document representations under a constant vector budget. We introduce four approaches for index compression: sequence resizing, memory tokens, hierarchical pooling, and a novel attention-guided clustering (AGC). AGC uses an attention-guided mechanism to identify the most semantically salient regions of a document as cluster centroids and to weight token aggregation. Evaluating these methods on retrieval tasks spanning text (BEIR), visual-document (ViDoRe), and video (MSR-VTT, MultiVENT 2.0), we show that attention-guided clustering consistently outperforms other parameterized compression methods (sequence resizing and memory tokens), provides greater flexibility in index size than non-parametric hierarchical clustering, and achieves competitive or improved performance compared to a full, uncompressed index. The source code is available at: github.com/hanxiangqin/omni-col-press.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
