arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2603.11631cs.AIcs.CV

VisDoT : 通过类人解释 grounding 和思维分解增强视觉推理

VisDoT : Enhancing Visual Reasoning through Human-Like Interpretation Grounding and Decomposition of Thought

Department of Computer Science and Artificial Intelligence, Dongguk University(东国大学计算机科学与人工智能系);Department of Electronics and Electrical Engineering, Dongguk University(东国大学电子与电气工程系)

Eunsoo Lee, Jeongwoo Lee, Minki Hong, Jangho Choi, Jihie Kim

AI总结:

VisDoT 通过类人解释 grounding 和思维分解提升视觉推理,显著提升图表问答和开放领域视觉问答性能。

摘要:

大型视觉-语言模型(LVLMs)在图表中可靠检测视觉基本元素并将其与语义表示对齐方面存在困难,这严重限制了其在复杂视觉推理上的性能。这种感知 grounding 的缺乏是图表推理的主要瓶颈。我们提出了 VisDoT,一个通过类人解释 grounding 增强视觉推理的框架。我们基于图形感知理论正式化了四个感知任务,包括位置和长度。在此基础上,我们引入了思维分解(DoT)提示,它依次将问题分解为视觉感知子问题和逻辑子问题。在 VisDoT 上微调 InternVL 在 ChartQA 上实现了 +11.2% 的提升,并在更具挑战性的 ChartQAPro 基准上超越了 GPT-4o。在新引入的 VisDoTQA 基准上,模型提升了 +33.2%。此外,一致的零样本收益在多样化的开放领域 VQA 基准上证实了感知-逻辑分离策略在视觉问答中的通用性。VisDoT 利用类人感知增强视觉 grounding,实现了最先进的图表理解和可解释的视觉推理。

英文摘要:

Large vision-language models (LVLMs) struggle to reliably detect visual primitives in charts and align them with semantic representations, which severely limits their performance on complex visual reasoning. This lack of perceptual grounding constitutes a major bottleneck for chart-based reasoning. We propose VisDoT, a framework that enhances visual reasoning through human-like interpretation grounding. We formalize four perceptual tasks based on the theory of graphical perception, including position and length. Building on this foundation, we introduce Decomposition-of-Thought (DoT) prompting, which sequentially separates questions into visual perception sub-questions and logic sub-questions. Fine-tuning InternVL with VisDoT achieves a +11.2% improvement on ChartQA and surpasses GPT-4o on the more challenging ChartQAPro benchmark. On the newly introduced VisDoTQA benchmark, the model improves by +33.2%. Furthermore, consistent zero-shot gains on diverse open-domain VQA benchmarks confirm the generalizability of the perception-logic separation strategy for visual question answering. VisDoT leverages human-like perception to enhance visual grounding, achieving state-of-the-art chart understanding and interpretable visual reasoning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑