MM-Matryoshka：通过二维多模态套娃训练框架实现预算弹性视觉文档检索
MM-Matryoshka: Towards Budget-Elastic Visual Document Retrieval via a 2D Multimodal Matryoshka Training Framework
AI总结：
提出MM-Matryoshka，一种二维套娃训练框架，使视觉文档检索器在向量维度和编码器深度上实现弹性预算选择，无需为不同预算训练独立模型。
摘要：
多向量视觉文档检索器通过深度视觉语言模型（VLM）为每个页面生成多个向量，实现强大的细粒度匹配，但这种设计在存储和计算开销上导致部署成本高昂。现有效率技术通常只优化预算的一部分，使得多模态检索器缺乏统一的方法来权衡精度与向量宽度和编码器深度。因此，我们提出MM-Matryoshka，一种用于预算弹性视觉文档检索（VDR）的二维套娃训练框架，使ColPali风格的多向量检索在维度和层两个方向上实现弹性。在推理时，单个检索器可以选择二维可调预算，无需为不同预算训练独立模型。通过在多个代表性骨干网络上的全面实验，我们证明MM-Matryoshka在显著降低存储和计算开销的同时，保留了比直接截断基线高得多的质量，从而为高效VDR提供了稳健的预算弹性。
英文摘要：
Multi-vector visual document retrievers achieve strong fine-grained matching by representing each page with multiple vectors from deep Vision-Language Models (VLMs), but this design makes deployment expensive in both storage and computational overhead. Existing efficiency techniques usually optimize only part of this budget, leaving multimodal retrievers without a unified way to trade accuracy for both vector width and encoder depth. Therefore, we propose MM-Matryoshka, a 2D Matryoshka training framework for budget-elastic Visual Document Retrieval (VDR), enabling ColPali-style multi-vector retrieval elastic along both dimension and layer. At inference time, a single retriever can select a 2D selectable budget without training separate models for different budgets. Through comprehensive experiments across multiple representative backbones, we demonstrate that by retaining significantly higher quality than direct truncation baselines while substantially reducing storage and computational overhead, MM-Matryoshka can offer robust budget elasticity for efficient VDR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
