RefChartQA：通过指令微调在图表图像上实现视觉答案定位
RefChartQA: Grounding Visual Answer on Chart Images through Instruction Tuning
AI总结：
针对图表理解缺乏视觉定位的问题，提出RefChartQA基准，通过指令微调5个VLM，利用空间感知将准确率提升超15%，并开源数据集与代码。
摘要：
近年来，视觉语言模型（VLMs）日益强调文档视觉定位，以实现更好的人机交互、可访问性和细节理解。然而，由于图表图像中视觉与数值关系交错的内在复杂性，其在图表等可视化方面的应用仍未得到充分探索。现有的图表理解方法主要侧重于回答问题，而不明确识别支持其预测的视觉元素。为弥补这一差距，我们引入了RefChartQA，这是一个新颖的基准，将图表问答（ChartQA）与视觉定位相结合，使模型能够引用图表图像中多个粒度的元素。此外，我们通过对5个最先进的视觉语言模型进行指令微调，在不同类别上进行了全面评估。我们的实验表明，通过定位引入空间感知可将响应准确率提高超过15%，减少幻觉，并提高模型可靠性。此外，我们识别了影响文本-空间对齐的关键因素，例如TinyChart中的架构改进，其利用令牌合并模块增强特征融合。我们的数据集已开源供社区开发和进一步改进。所有模型和代码将在https://github.com/moured/RefChartQA上公开提供。
英文摘要：
Recently, Vision Language Models (VLMs) have increasingly emphasized document visual grounding to achieve better human-computer interaction, accessibility, and detailed understanding. However, its application to visualizations such as charts remains under-explored due to the inherent complexity of interleaved visual-numerical relationships in chart images. Existing chart understanding methods primarily focus on answering questions without explicitly identifying the visual elements that support their predictions. To bridge this gap, we introduce RefChartQA, a novel benchmark that integrates Chart Question Answering (ChartQA) with visual grounding, enabling models to refer elements at multiple granularities within chart images. Furthermore, we conduct a comprehensive evaluation by instruction-tuning 5 state-of-the-art VLMs across different categories. Our experiments demonstrate that incorporating spatial awareness via grounding improves response accuracy by over 15%, reducing hallucinations, and improving model reliability. Additionally, we identify key factors influencing text-spatial alignment, such as architectural improvements in TinyChart, which leverages a token-merging module for enhanced feature fusion. Our dataset is open-sourced for community development and further advancements. All models and code will be publicly available at https://github.com/moured/RefChartQA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
