arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.17441cs.CVcs.AIcs.CLcs.DLcs.SE
收藏

让流程图图像变得机器可解释

Towards Making Flowchart Images Machine Interpretable

IIT Jodhpur(焦特布尔印度理工学院)

Shreya Shukla, Prajwal Gatti, Yogesh Kumar, Vikash Yadav, Anand Mishra

AI总结:

本文提出基于Transformer的FloCo-T5框架,将流程图图像转换为可执行Python代码,并引入包含11,884张图像的FloCo数据集,实验证明其在代码生成指标上优于竞争基线。

摘要:

计算机编程教科书和软件文档中经常包含流程图,用于说明算法或过程。现代OCR引擎通常将这些流程图标记为图形,并在后续处理中忽略它们。在本文中,我们致力于通过将流程图图像转换为可执行的Python代码,使其成为机器可解释的。为此,受自然语言到代码生成文献中近期成功的启发,我们提出了一种新颖的基于Transformer的框架,即FloCo-T5。我们的模型非常适合此任务,因为它能有效地学习编程语言的语义、结构和模式,并利用这些来生成语法正确的代码。我们还使用了一个任务特定的预训练目标,通过大量保持逻辑的增强代码样本来预训练FloCo-T5。此外,为了对这一问题进行严谨的研究,我们引入了FloCo数据集,其中包含11,884张流程图图像及其对应的Python代码。我们的实验显示了有希望的结果,FloCo-T5在代码生成指标上明显优于相关的竞争基线。我们公开了我们的数据集和实现。

英文摘要:

Computer programming textbooks and software documentations often contain flowcharts to illustrate the flow of an algorithm or procedure. Modern OCR engines often tag these flowcharts as graphics and ignore them in further processing. In this paper, we work towards making flowchart images machine-interpretable by converting them to executable Python codes. To this end, inspired by the recent success in natural language to code generation literature, we present a novel transformer-based framework, namely FloCo-T5. Our model is well-suited for this task,as it can effectively learn semantics, structure, and patterns of programming languages, which it leverages to generate syntactically correct code. We also used a task-specific pre-training objective to pre-train FloCo-T5 using a large number of logic-preserving augmented code samples. Further, to perform a rigorous study of this problem, we introduce theFloCo dataset that contains 11,884 flowchart images and their corresponding Python codes. Our experiments show promising results, and FloCo-T5 clearly outperforms related competitive baselines on code generation metrics. We make our dataset and implementation publicly available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑