ChartM$^3$：多模态指令图表编辑基准
ChartM$^3$: Benchmarking Chart Editing with Multimodal Instructions
AI总结：
本文提出多模态图表编辑范式，并构建基准ChartM$^3$（含1000样本、四级难度）及训练集Chart$\ ext{M}^3$-Train（24000样本），发现现有MLLMs局限，微调后显著提升编辑性能。
摘要：
图表是一种基础的可视化格式，在研究和工业界的数据分析中被广泛使用。虽然基于高层意图编辑图表具有很大的实用价值，但现有方法主要依赖自然语言指令，这些指令往往过于模糊，难以支持细粒度的编辑。在这项工作中，我们提出了一种新颖的多模态图表编辑范式，其中用户意图通过自然语言和视觉指示的组合来表达，视觉指示明确突出要修改的元素。为了支持这一范式，我们提出了Chart$\ ext{M}^3$，一个新的基准，用于多模态图表编辑，具有多级复杂度和多角度评估。Chart$\ ext{M}^3$包含1,000个样本，涵盖四个编辑难度级别。每个样本包括（图表，代码，多模态指令）形式的三元组。为了全面评估图表编辑模型，Chart$\ ext{M}^3$提供了评估视觉外观和代码正确性的指标。我们的基准揭示了当前多模态大语言模型（MLLMs）的显著局限性，包括GPT-4o，特别是在解释和响应视觉指示方面的能力。为了解决这个问题，我们构建了Chart$\ ext{M}^3$-Train，一个包含24,000个多模态图表编辑样本的大规模训练集。在此数据集上微调MLLMs带来了显著的改进，证明了多模态监督在构建实用图表编辑系统中的重要性。我们的数据集、代码和评估工具可在https://github.com/MLrollIT/ChartM3获取。
英文摘要：
Charts are a fundamental visualization format widely used in data analysis across research and industry. While enabling users to edit charts based on high-level intentions is of great practical value, existing methods primarily rely on natural language instructions, which are often too ambiguous to support fine-grained editing. In this work, we introduce a novel paradigm for multimodal chart editing, where user intent is expressed through a combination of natural language and visual indicators that explicitly highlight the elements to be modified. To support this paradigm, we present Chart$\text{M}^3$, a new benchmark for Multimodal chart editing with Multi-level complexity and Multi-perspective evaluation. Chart$\text{M}^3$ contains 1,000 samples spanning four levels of editing difficulty. Each sample includes triplets in the form of (chart, code, multimodal instructions). To comprehensively evaluate chart editing models, Chart$\text{M}^3$ provides metrics that assess both visual appearance and code correctness. Our benchmark reveals significant limitations in current multimodal large language models (MLLMs), including GPT-4o, particularly in their ability to interpret and act on visual indicators. To address this, we construct Chart$\text{M}^3$-Train, a large-scale training set with 24,000 multimodal chart editing samples. Fine-tuning MLLMs on this dataset leads to substantial improvements, demonstrating the importance of multimodal supervision in building practical chart editing systems. Our datasets, codes, and evaluation tools are available at https://github.com/MLrollIT/ChartM3. %https://github.com/MLrollIT/ChartM3Our datasets, codes, and evaluation tools are available at https://github.com/yaolinli/VCE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
