ChartEditBench: 评估多轮视觉引导图表编辑在多模态语言模型中的表现
ChartEditBench: Evaluating Grounded Multi-Turn Chart Editing in Multimodal Language Models
AI总结：
ChartEditBench通过代码进行多轮视觉引导图表编辑，评估多模态语言模型在持续、上下文感知编辑中的表现，揭示了多轮交互中错误累积和共享上下文失效的问题。
摘要：
尽管多模态大语言模型（MLLMs）在单轮图表生成任务上表现优异，但其在支持真实世界探索性数据分析能力仍缺乏深入研究。在实践中，用户通过多轮交互逐步优化可视化，这一过程需要维持共同语境、跟踪先前编辑并适应不断变化的偏好。我们引入了ChartEditBench，这是一个通过代码进行增量、视觉引导图表编辑的基准测试集，包含5000个难度可控的修改链和一个经过严格人工验证的子集。与以往的一次性基准测试不同，ChartEditBench评估的是持续、上下文感知的编辑能力。我们进一步提出了一种稳健的评估框架，通过整合执行基于的准确性检查、像素级视觉相似性和逻辑代码验证来弥补LLM-as-a-Judge指标的局限性。对最先进的MLLMs的实验表明，在多轮设置中由于错误累积和共享上下文的失效，性能显著下降，尽管在风格化编辑上表现强劲，但在数据导向的转换上却频繁出现执行失败。ChartEditBench为基于意图的多模态编程建立了具有挑战性的测试平台。
英文摘要：
While Multimodal Large Language Models (MLLMs) perform strongly on single-turn chart generation, their ability to support real-world exploratory data analysis remains underexplored. In practice, users iteratively refine visualizations through multi-turn interactions that require maintaining common ground, tracking prior edits, and adapting to evolving preferences. We introduce ChartEditBench, a benchmark for incremental, visually grounded chart editing via code, comprising 5,000 difficulty-controlled modification chains and a rigorously human-verified subset. Unlike prior one-shot benchmarks, ChartEditBench evaluates sustained, context-aware editing. We further propose a robust evaluation framework that mitigates limitations of LLM-as-a-Judge metrics by integrating execution-based fidelity checks, pixel-level visual similarity, and logical code verification. Experiments with state-of-the-art MLLMs reveal substantial degradation in multi-turn settings due to error accumulation and breakdowns in shared context, with strong performance on stylistic edits but frequent execution failures on data-centric transformations. ChartEditBench, establishes a challenging testbed for grounded, intent-aware multimodal programming.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
