SBS Figures:基于分阶段合成图像的图表面QA预训练
SBS Figures: Pre-training Figure QA from Stage-by-Stage Synthesized Images
AI总结:
提出SBSFigures数据集,通过分阶段合成流程高效生成带完整标注的图表QA数据,解决LLM直接生成图表的代码错误和内容重复问题,并展现出强大的预训练效果,仅需少量真实数据即可实现高效训练。
摘要:
构建大规模图表面QA数据集需要大量工作,包括收集和选择图表、提取文本、数字和颜色等属性,以及生成问答对。尽管LLM的最新发展推动了图表合成的尝试,但大多数工作主要集中在QA生成上。此外,直接使用LLM创建图表常遇到代码错误、图表外观相似以及内容重复等问题。为解决此问题,我们提出了SBSFigures(Stage-by-Stage Synthetic Figures),一个用于预训练图表面QA的数据集。我们提出的流程能够创建带有可视化数据完整标注和密集QA标注的图表,且无需任何人工标注过程。我们的分阶段流程使得高效创建多样化主题和外观的图表成为可能,同时将代码错误降至最低。我们的SBSFigures展现了强大的预训练效果,使得从我们的预训练权重出发,仅需少量真实图表数据即可实现高效训练。
英文摘要:
Building a large-scale figure QA dataset requires a considerable amount of work, from gathering and selecting figures to extracting attributes like text, numbers, and colors, and generating QAs. Although recent developments in LLMs have led to efforts to synthesize figures, most of these focus primarily on QA generation. Additionally, creating figures directly using LLMs often encounters issues such as code errors, similar-looking figures, and repetitive content in figures. To address this issue, we present SBSFigures (Stage-by-Stage Synthetic Figures), a dataset for pre-training figure QA. Our proposed pipeline enables the creation of chart figures with complete annotations of the visualized data and dense QA annotations without any manual annotation process. Our stage-by-stage pipeline makes it possible to create diverse topic and appearance figures efficiently while minimizing code errors. Our SBSFigures demonstrate a strong pre-training effect, making it possible to achieve efficient training with a limited amount of real-world chart data starting from our pre-trained weights.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。