MSG-Chart：用于 ChartQA 的多模态场景图
MSG-Chart: Multimodal Scene Graph for ChartQA
AI总结：
本文提出 MSG-Chart，通过视觉图与文本图构成的多模态场景图显式建模图表元素及数据模式关系，并可作为归纳偏置集成到 vision transformer，提升 ChartQA 与 OpenCQA 性能。
摘要：
自动图表问答（ChartQA）具有挑战性，原因在于图表元素分布复杂，且底层数据的模式并未在图表中显式呈现。为应对这一挑战，我们设计了一种面向图表的联合多模态场景图，以显式表示图表元素及其模式之间的关系。所提出的多模态场景图包含视觉图和文本图，用于共同捕获图表中的结构知识与语义知识。该图模块可作为归纳偏置，方便地集成到不同的 vision transformer 中。实验表明，加入所提出的图模块能够增强对图表元素结构和语义的理解，从而在公开基准 ChartQA 和 OpenCQA 上提升性能。
英文摘要：
Automatic Chart Question Answering (ChartQA) is challenging due to the complex distribution of chart elements with patterns of the underlying data not explicitly displayed in charts. To address this challenge, we design a joint multimodal scene graph for charts to explicitly represent the relationships between chart elements and their patterns. Our proposed multimodal scene graph includes a visual graph and a textual graph to jointly capture the structural and semantical knowledge from the chart. This graph module can be easily integrated with different vision transformers as inductive bias. Our experiments demonstrate that incorporating the proposed graph module enhances the understanding of charts' elements' structure and semantics, thereby improving performance on publicly available benchmarks, ChartQA and OpenCQA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
