ChartAgent: 一种用于复杂图表问答中视觉基础推理的多模态智能体
ChartAgent: A Multimodal Agent for Visually Grounded Reasoning in Complex Chart Question Answering
AI总结：
提出ChartAgent框架，通过迭代分解查询为视觉子任务并利用图表专用视觉工具（如绘制注释、裁剪区域）进行空间域推理，在ChartBench和ChartX上取得最先进性能，尤其对无标注图表提升显著。
摘要：
最近的多模态大语言模型在基于图表的视觉问答中显示出潜力，但在无标注图表上——即那些需要精确视觉解释而非依赖文本捷径的图表——其性能急剧下降。为了解决这个问题，我们引入了ChartAgent，一种新颖的智能体框架，它直接在图表的空间域内显式执行视觉推理。与文本思维链推理不同，ChartAgent通过专门的行动（如绘制注释、裁剪区域（例如分割饼图切片、隔离条形图）和定位坐标轴）迭代地将查询分解为视觉子任务，并主动操作和交互图表图像，使用图表专用视觉工具库来完成每个子任务。这种迭代推理过程密切模仿了人类理解图表的认知策略。ChartAgent在ChartBench和ChartX基准测试上达到了最先进的准确率，整体上比先前方法绝对提升高达16.07%，在无标注、数值密集的查询上提升17.31%。此外，我们的分析表明，ChartAgent (a) 在多种图表类型上有效，(b) 在不同视觉和推理复杂度水平上均取得最高分数，(c) 作为一个即插即用的框架，提升了多种基础LLM的性能。我们的工作是首批使用工具增强的多模态智能体展示图表理解中视觉基础推理的工作之一。
英文摘要：
Recent multimodal LLMs have shown promise in chart-based visual question answering, but their performance declines sharply on unannotated charts-those requiring precise visual interpretation rather than relying on textual shortcuts. To address this, we introduce ChartAgent, a novel agentic framework that explicitly performs visual reasoning directly within the chart's spatial domain. Unlike textual chain-of-thought reasoning, ChartAgent iteratively decomposes queries into visual subtasks and actively manipulates and interacts with chart images through specialized actions such as drawing annotations, cropping regions (e.g., segmenting pie slices, isolating bars), and localizing axes, using a library of chart-specific vision tools to fulfill each subtask. This iterative reasoning process closely mirrors human cognitive strategies for chart comprehension. ChartAgent achieves state-of-the-art accuracy on the ChartBench and ChartX benchmarks, surpassing prior methods by up to 16.07% absolute gain overall and 17.31% on unannotated, numerically intensive queries. Furthermore, our analyses show that ChartAgent is (a) effective across diverse chart types, (b) achieves the highest scores across varying visual and reasoning complexity levels, and (c) serves as a plug-and-play framework that boosts performance across diverse underlying LLMs. Our work is among the first to demonstrate visually grounded reasoning for chart understanding using tool-augmented multimodal agents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
