arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.04142cs.CVcs.AI
收藏

JSynFlow:利用大语言模型构建的日本合成流程图视觉问答数据集

JSynFlow: Japanese Synthesised Flowchart Visual Question Answering Dataset built with Large Language Models

The Japan Research Institute, Limited(日本研究所有限公司)

Hiroshi Sasaki

AI总结:

JSynFlow利用大语言模型生成日本流程图视觉问答数据集,提升VLM在流程图问答任务中的性能。

摘要:

视觉和语言模型(VLMs)被期望通过问答接口分析复杂的文档,如包含流程图的文档。能够识别和解释这些流程图的能力需求很高,因为它们提供了文本解释中不可获得的有价值见解。然而,开发具有精确流程图理解能力的VLMs需要大量包含流程图图像及其相应文本的大型数据集,其创建过程非常耗时。为了解决这一挑战,我们引入了JSynFlow,一个利用大语言模型(LLMs)生成的日本流程图视觉问答合成数据集。该数据集包含各种职业的任务描述、由领域特定语言(DSL)代码渲染的相应流程图图像以及相关的问答对。本文详细介绍了数据集的合成过程,并展示了使用JSynFlow微调显著提高了VLM在基于流程图的问答任务上的性能。我们的数据集可在https://huggingface.co/datasets/jri-advtechlab/jsynflow公开获取。

英文摘要:

Vision and language models (VLMs) are expected to analyse complex documents, such as those containing flowcharts, through a question-answering (QA) interface. The ability to recognise and interpret these flowcharts is in high demand, as they provide valuable insights unavailable in text-only explanations. However, developing VLMs with precise flowchart understanding requires large-scale datasets of flowchart images and corresponding text, the creation of which is highly time-consuming. To address this challenge, we introduce JSynFlow, a synthesised visual QA dataset for Japanese flowcharts, generated using large language models (LLMs). Our dataset comprises task descriptions for various business occupations, the corresponding flowchart images rendered from domain-specific language (DSL) code, and related QA pairs. This paper details the dataset's synthesis procedure and demonstrates that fine-tuning with JSynFlow significantly improves VLM performance on flowchart-based QA tasks. Our dataset is publicly available at https://huggingface.co/datasets/jri-advtechlab/jsynflow.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑