arXiv 2507.124902025-07-15cs.CVcs.AIcs.CLcs.LG

面向文档视觉问答的视觉语言模型中的空间定位解释

Spatially Grounded Explanations in Vision Language Models for Document Visual Question Answering

Computer Vision Center, Universitat Autònoma de Barcelona（计算机视觉中心，巴塞罗那自治大学）；Gradiant

Maximiliano Hormazábal Lagos, Héctor Cerezo-Costas, Dimosthenis Karatzas

备注：: This work has been accepted for presentation at the 16th Conference and Labs of the Evaluation Forum (CLEF 2025) and will be published in the proceedings by Springer in the Lecture Notes in Computer Science (LNCS) series. Please cite the published version when available

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出无需训练的模型无关流程EaGERS，通过生成推理依据、空间定位相关区域并限制响应生成，在DocVQA上提升准确率与可解释性。

摘要：

我们提出EaGERS，一个完全无需训练且与模型无关的流程，该流程（1）通过视觉语言模型生成自然语言推理依据，（2）通过在可配置网格上计算多模态嵌入相似度并进行多数投票，将这些推理依据定位到空间子区域，（3）仅从掩码图像中选定的相关区域限制响应的生成。在DocVQA数据集上的实验表明，我们的最佳配置不仅在精确匹配准确率和平均归一化Levenshtein相似度指标上优于基础模型，而且在不进行额外模型微调的情况下增强了DocVQA的透明性和可复现性。

英文摘要：

We introduce EaGERS, a fully training-free and model-agnostic pipeline that (1) generates natural language rationales via a vision language model, (2) grounds these rationales to spatial sub-regions by computing multimodal embedding similarities over a configurable grid with majority voting, and (3) restricts the generation of responses only from the relevant regions selected in the masked image. Experiments on the DocVQA dataset demonstrate that our best configuration not only outperforms the base model on exact match accuracy and Average Normalized Levenshtein Similarity metrics but also enhances transparency and reproducibility in DocVQA without additional model fine-tuning.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。