ViInfographicVQA:单图和多图越南信息图视觉问答基准测试
ViInfographicVQA: A Benchmark for Single and Multi-image Visual Question Answering on Vietnamese Infographics
AI总结:
ViInfographicVQA是首个针对越南信息图VQA的基准测试,通过单图和多图任务评估模型在复杂信息图中的视觉问答能力。
摘要:
信息图视觉问答(InfographicVQA)评估模型在阅读和推理数据丰富、布局密集的视觉内容方面的能力,这些内容结合了文本、图表、图标和设计元素。与场景文本或自然图像VQA相比,信息图需要更强的OCR、布局理解以及数值和语义推理能力。我们介绍了ViInfographicVQA,这是首个针对越南信息图VQA的基准测试,包含超过6747个真实世界信息图和20409个由人类验证的问题-答案对,涵盖经济、医疗、教育等领域。该基准测试包含两种评估设置。单图任务遵循传统设置,每个问题使用单个信息图回答。多图任务需要跨多个语义相关的信息图综合证据,据我们所知,这是首个针对越南的跨图推理评估。我们在此基准测试上评估了多种最近的视觉-语言模型,揭示了显著的性能差异,其中最大的错误发生在需要跨图整合和非跨度推理的多图问题上。ViInfographicVQA为越南信息图VQA提供了基准测试结果,并揭示了当前多模态模型在低资源环境中的局限性,鼓励未来对布局感知和跨图推理方法的探索。
英文摘要:
Infographic Visual Question Answering (InfographicVQA) evaluates a model's ability to read and reason over data-rich, layout-heavy visuals that combine text, charts, icons, and design elements. Compared with scene-text or natural-image VQA, infographics require stronger integration of OCR, layout understanding, and numerical and semantic reasoning. We introduce ViInfographicVQA, the first benchmark for Vietnamese InfographicVQA, comprising over 6747 real-world infographics and 20409 human-verified question-answer pairs across economics, healthcare, education, and more. The benchmark includes two evaluation settings. The Single-image task follows the traditional setup in which each question is answered using a single infographic. The Multi-image task requires synthesizing evidence across multiple semantically related infographics and is, to our knowledge, the first Vietnamese evaluation of cross-image reasoning in VQA. We evaluate a range of recent vision-language models on this benchmark, revealing substantial performance disparities, with the most significant errors occurring on Multi-image questions that involve cross-image integration and non-span reasoning. ViInfographicVQA contributes benchmark results for Vietnamese InfographicVQA and sheds light on the limitations of current multimodal models in low-resource contexts, encouraging future exploration of layout-aware and cross-image reasoning methods.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。