ReAlign：通过推理引导的细粒度对齐优化视觉文档检索
ReAlign: Optimizing the Visual Document Retriever with Reasoning-Guided Fine-Grained Alignment
AI总结：
本文提出ReAlign方法，利用VLM推理能力生成细粒度视觉描述作为监督信号，提升视觉文档检索性能，实验表明在不同领域数据集上均取得2%的相对提升。
摘要：
视觉文档检索旨在从视觉丰富的文档集合中检索与查询相关的文档页面。现有方法通常使用视觉语言模型（VLMs）将查询和视觉页面编码到共享嵌入空间，并通过对比训练进行优化。然而，在视觉文档表示中，局部证据通常分散在复杂的文档布局中，使检索模型难以捕捉关键线索进行有效的嵌入学习。本文提出推理引导对齐（ReAlign），通过利用VLMs的推理能力，生成细粒度的视觉文档描述作为训练的监督信号。具体而言，ReAlign采用高性能的VLM识别页面上的查询相关区域，然后生成基于裁剪视觉区域的查询感知描述。检索器随后使用这些区域聚焦描述进行训练，通过鼓励由区域聚焦描述诱导的文档排序分布与由原始查询诱导的分布相匹配，实现查询与视觉文档语义的对齐。在多样化的视觉丰富文档检索基准测试中，实验表明ReAlign在领域内和领域外数据集上均提升了视觉文档检索性能，达到最高2%的相对提升。此外，ReAlign的优势能够跨不同的VLM骨干模型泛化，通过引导模型更好地聚焦于文档表示的关键视觉线索。所有代码和数据集均可在https://github.com/NEUIR/ReAlign获取。
英文摘要：
Visual document retrieval aims to retrieve a set of document pages relevant to a query from visually rich collections. Existing methods often employ Vision-Language Models (VLMs) to encode queries and visual pages into a shared embedding space, which is then optimized via contrastive training. However, during visual document representation, localized evidence is usually scattered across complex document layouts, making it difficult for retrieval models to capture crucial cues for effective embedding learning. In this paper, we propose Reasoning-Guided Alignment (ReAlign), a method that enhances visual document retrieval by leveraging the reasoning capability of VLMs to provide fine-grained visual document descriptions as supervision signals for training. Specifically, ReAlign employs a superior VLM to identify query-related regions on a page and then generates a query-aware description grounding the cropped visual regions. The retriever is then trained using these region-focused descriptions to align the semantics between queries and visual documents by encouraging the document ranking distribution induced by the region-focused descriptions to match that induced by the original query. Experiments on diverse visually rich document retrieval benchmarks demonstrate that ReAlign consistently improves visual document retrieval performance on both in-domain and out-of-domain datasets, achieving up to 2% relative improvements. Moreover, the advantages of ReAlign generalize across different VLM backbones by guiding models to better focus their attention on critical visual cues for document representation. All code and datasets are available at https://github.com/NEUIR/ReAlign.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
