从少量数据中学习更多：利用反事实以提高图表理解的数据效率
Learning More from Less: Exploiting Counterfactuals for Data-Efficient Chart Understanding
AI总结：
本文提出ChartCF框架，通过反事实数据合成和多模态偏好优化，提升图表理解的反事实敏感性，实验表明其在少数据下表现优异。
摘要：
视觉-语言模型（VLMs）在图表理解中取得了显著进展，主要得益于对日益庞大的合成数据集的监督微调（SFT）。然而，仅扩大SFT数据量是低效的，并忽略了图表的关键特性：图表是程序生成的视觉艺术，其中小的代码控制的视觉变化可导致语义和正确答案的剧烈变化。学习这种反事实敏感性需要VLMs能够区分细粒度的视觉差异，但标准SFT将训练实例独立处理，提供有限的监督以强制这种行为。为此，我们引入ChartCF，一种数据高效的训练框架，旨在增强反事实敏感性。ChartCF包括：（1）通过代码修改生成反事实数据的管道，（2）基于图表相似性的数据选择策略，用于过滤过于困难的样本以提高训练效率，以及（3）跨文本和视觉模态的多模态偏好优化。在五个基准测试中，实验表明ChartCF在使用显著较少训练数据的情况下，实现了优于或与强大图表专用VLMs相当的性能。
英文摘要：
Vision-Language Models (VLMs) have demonstrated remarkable progress in chart understanding, largely driven by supervised fine-tuning (SFT) on increasingly large synthetic datasets. However, scaling SFT data alone is inefficient and overlooks a key property of charts: charts are programmatically generated visual artifacts, where small, code-controlled visual changes can induce drastic shifts in semantics and correct answers. Learning this counterfactual sensitivity requires VLMs to discriminate fine-grained visual differences, yet standard SFT treats training instances independently and provides limited supervision to enforce this behavior. To address this, we introduce ChartCF, a data-efficient training framework designed to enhance counterfactual sensitivity. ChartCF consists of: (1) a counterfactual data synthesis pipeline via code modification, (2) a chart similarity-based data selection strategy that filters overly difficult samples for improved training efficiency, and (3) multimodal preference optimization across both textual and visual modalities. Experiments on five benchmarks show that ChartCF achieves superior or comparable performance to strong chart-specific VLMs while using significantly less training data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
