ChartCap：缓解密集图表描述中的幻觉
ChartCap: Mitigating Hallucination of Dense Chart Captioning
AI总结：
针对图表描述中的幻觉问题，提出包含565K真实图表及密集描述的ChartCap数据集，并设计四阶段生成流水线与视觉一致性分数，微调模型后显著提升描述准确性和信息量，减少幻觉。
摘要：
为图表生成准确、信息丰富且无幻觉的描述，对视觉语言模型而言仍具挑战性，主要原因是缺乏大规模、高质量的真实世界图表数据集。然而，现有的真实世界图表数据集存在包含无法从图表中推断出的无关信息，以及未能充分捕捉结构元素和关键见解的问题。因此，我们引入了ChartCap，一个包含565K张真实世界图表图像的大规模数据集，每张图像配有类型特定的密集描述，这些描述排除了无关信息，并详细突出了结构元素和关键见解。为了构建ChartCap，我们设计了一个四阶段流水线，仅使用图表中可辨别的数据生成描述，并采用基于循环一致性的人工验证，在不牺牲准确性的前提下加速质量控制。此外，我们提出了一种新颖的指标——视觉一致性分数，通过衡量从描述重新生成的图表与原始图表之间的相似性来评估描述质量，该指标独立于参考描述。大量实验证实，在ChartCap上微调的模型能够持续生成更准确、信息更丰富且幻觉更少的描述，超越了开源和专有模型，甚至优于人工标注的描述。
英文摘要：
Generating accurate, informative, and hallucination-free captions for charts remains challenging for vision language models, primarily due to the lack of large-scale, high-quality datasets of real-world charts. However, existing real-world chart datasets suffer from the inclusion of extraneous information that cannot be inferred from the chart and failure to sufficiently capture structural elements and key insights. Therefore, we introduce ChartCap, a large-scale dataset of 565K real-world chart images paired with type-specific, dense captions that exclude extraneous information and highlight both structural elements and key insights in detail. To build ChartCap, we design a four-stage pipeline that generates captions using only the discernible data from the chart and employ a cycle consistency-based human verification, which accelerates quality control without sacrificing accuracy. Additionally, we propose a novel metric, the Visual Consistency Score, which evaluates caption quality by measuring the similarity between the chart regenerated from a caption and the original chart, independent of reference captions. Extensive experiments confirms that models fine-tuned on ChartCap consistently generate more accurate and informative captions with reduced hallucinations, surpassing both open-source and proprietary models and even human-annotated captions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
