SciCap+：用于研究科学图表字幕生成挑战的知识增强数据集
SciCap+: A Knowledge Augmented Dataset to Study the Challenges of Scientific Figure Captioning
AI总结：
本文构建含提及段落与 OCR 标记的 SciCap+ 知识增强数据集，并以 M4C-Captioner 为基线验证跨模态知识可显著提升科学图表字幕生成效果。
摘要：
在学术文档中，图表为向读者传达科学发现提供了一种直观方式。自动化生成图表字幕有助于推动模型对科学文档的理解超越文本层面，并帮助作者撰写信息丰富的字幕，从而促进科学发现的交流。与以往研究不同，我们将科学图表字幕生成重新定义为一项知识增强的图像字幕生成任务，模型需要利用跨模态嵌入的知识来生成字幕。为此，我们将大规模 SciCap 数据集扩展为 SciCap+，其中包含提及段落（提及图表的段落）和 OCR 标记。随后，我们使用 M4C-Captioner（一种基于多模态 transformer 并带有指针网络的模型）作为本研究的基线开展实验。结果表明，提及段落可作为额外的上下文知识；与仅使用图表的基线相比，它显著提升了图像字幕自动标准评价分数。人工评价进一步揭示了生成对读者具有信息量的图表字幕所面临的挑战。代码和 SciCap+ 数据集将公开发布于 https://github.com/ZhishenYang/scientific_figure_captioning_dataset。
英文摘要：
In scholarly documents, figures provide a straightforward way of communicating scientific findings to readers. Automating figure caption generation helps move model understandings of scientific documents beyond text and will help authors write informative captions that facilitate communicating scientific findings. Unlike previous studies, we reframe scientific figure captioning as a knowledge-augmented image captioning task that models need to utilize knowledge embedded across modalities for caption generation. To this end, we extended the large-scale SciCap dataset~\cite{hsu-etal-2021-scicap-generating} to SciCap+ which includes mention-paragraphs (paragraphs mentioning figures) and OCR tokens. Then, we conduct experiments with the M4C-Captioner (a multimodal transformer-based model with a pointer network) as a baseline for our study. Our results indicate that mention-paragraphs serves as additional context knowledge, which significantly boosts the automatic standard image caption evaluation scores compared to the figure-only baselines. Human evaluations further reveal the challenges of generating figure captions that are informative to readers. The code and SciCap+ dataset will be publicly available at https://github.com/ZhishenYang/scientific_figure_captioning_dataset
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
