DRISHTIKON：文档中多粒度的视觉定位
DRISHTIKON: Visual Grounding at Multiple Granularities in Documents
AI总结：
提出多粒度多区块视觉定位框架DRISHTIKON，结合多语言OCR与大语言模型实现区块至点级别的答案定位，并构建MGVG基准，实验证明其在精确率和召回率上达到最优。
摘要：
文本丰富的文档图像中的视觉定位是文档智能和视觉问答（VQA）系统面临的一个关键但尚未充分探索的挑战。我们提出了DRISHTIKON，这是一个多粒度、多区块的视觉定位框架，旨在增强复杂、多语言文档VQA的可解释性和可信度。我们的方法集成了多语言OCR、大语言模型和一种新颖的区域匹配算法，以在区块、行、词和点级别定位答案跨度。我们引入了多粒度视觉定位（MGVG）基准，这是一个由来自不同领域的各种通知组成的精选测试集，每个通知都手动标注了跨多个粒度的细粒度、人工验证标签。大量实验表明，我们的方法实现了最先进的定位精度，其中行级粒度在精确率和召回率之间提供了最佳平衡。消融研究进一步突出了多区块和多行推理的优势。比较评估显示，领先的视觉语言模型在精确定位方面存在困难，这凸显了我们基于结构化对齐方法的有效性。我们的发现为在具有多粒度定位支持的现实世界、以文本为中心的场景中构建更强大、更可解释的文档理解系统铺平了道路。代码和数据集已公开供未来研究使用。
英文摘要：
Visual grounding in text-rich document images is a critical yet underexplored challenge for Document Intelligence and Visual Question Answering (VQA) systems. We present DRISHTIKON, a multi-granular and multi-block visual grounding framework designed to enhance interpretability and trust in VQA for complex, multilingual documents. Our approach integrates multilingual OCR, large language models, and a novel region matching algorithm to localize answer spans at the block, line, word, and point levels. We introduce the Multi-Granular Visual Grounding (MGVG) benchmark, a curated test set of diverse circular notifications from various sectors, each manually annotated with fine-grained, human-verified labels across multiple granularities. Extensive experiments show that our method achieves state-of-the-art grounding accuracy, with line-level granularity providing the best balance between precision and recall. Ablation studies further highlight the benefits of multi-block and multi-line reasoning. Comparative evaluations reveal that leading vision-language models struggle with precise localization, underscoring the effectiveness of our structured, alignment-based approach. Our findings pave the way for more robust and interpretable document understanding systems in real-world, text-centric scenarios with multi-granular grounding support. Code and dataset are made available for future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
