超越图像块：通过文本监督学习全局布局用于晚期交互视觉文档检索
Beyond Bag-of-Patches: Learning Global Layout via Textual Supervision for Late-Interaction Visual Document Retrieval
AI总结：
本文提出通过文本监督学习文档全局布局，改进视觉文档检索的晚期交互架构，提升检索性能。
摘要：
视觉文档检索（VDR）模型大多依赖晚期交互架构，其中文档通过一组局部块嵌入表示，然后与查询词匹配。尽管高效，这种架构优先考虑局部相似性而非文档的全局布局结构来估计文档与查询的相关性。在实践中，这导致错误，因为相关性来源于具有异构布局（包含图表和文本）的文档布局结构。我们提出一种多模态编码器，通过文本描述编码文档布局信息来增强局部块表示，从而学习文档布局。在四个ViDoRe-v2数据集上，我们的模型在nDCG@5和MAP@5指标上分别优于最强大的架构相当的ColPali/ColQwen基线，且在每个数据集上均具有统计显著性提升。
英文摘要：
Visual Document Retrieval (VDR) models mostly rely on late interaction architectures, in which documents are represented by a set of local patch embeddings and then matched against query tokens. While efficient, this architecture prioritizes local similarity over global layout structure of documents to estimate relevancy between documents and query. In practice, this leads to errors as relevance originates from layout structure of documents with heterogeneous layouts combining figures, tables, and text. We make document layout learnable without changing inference. We propose a multimodal encoder that augments local patch representations with a global layout embedding, trained via textual descriptions encoding document layout information. Across four ViDoRe-v2 datasets, our model improves over the strongest architecturally comparable ColPali/ColQwen baseline by +2.4 nDCG@5 and +2.3 MAP@5, with statistically significant per-dataset gains over ColQwen.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
