arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2112.08259cs.DB
收藏

or2yw:将OpenRefineHistories建模并可视化为YesWorkflow图

or2yw: Modeling and Visualizing OpenRefineHistories as YesWorkflow Diagrams

Nikolaus Nova Parulian, Lan Li, Bertram Ludaescher

摘要:

OpenRefine是一个流行的开放源数据清理工具。它允许用户以JSON格式导出先前执行的数据清理工作流,以便在其他数据集上重用。我们已经开发了or2yw,一个新的工具,将JSON格式的OpenRefine操作历史映射到YesWorkflow(YW)模型,然后可以使用YW工具进行可视化和查询。后者最初是为了让研究人员能够以一种简单的方式注释他们的程序脚本,以揭示这些脚本中隐含的工作流程步骤和可重用依赖关系。使用or2yw,用户可以从OpenRefine操作历史中自动生成YW模型,从而提供先前执行的数据清理操作序列的“工作流视图”。or2yw工具可以生成不同类型的YesWorkflow模型,例如,反映OpenRefine中操作的顺序执行次序的线性模型,以及基于步骤之间的依赖性的简单分析揭示独立工作流分支的\emph{parallel model}:如果两个操作彼此独立(例如,当它们读取和写入的列不重叠时),则这些可以被视为数据清理工作流中的并行步骤。由此产生的YW模型可以被理解为一种形式的预期出处,即,可以查询和可视化的知识工件(i)以帮助作者记录他们自己的数据清理工作流,从而增加透明度,以及(ii)帮助可能想要重用这些工作流的其他用户更好地理解它们。

英文摘要:

OpenRefine is a popular open-source data cleaning tool. It allows users to export a previously executed data cleaning workflow in a JSON format for possible reuse on other datasets. We have developed or2yw, a novel tool that maps a JSON-formatted OpenRefine operation history to a YesWorkflow (YW) model, which then can be visualized and queried using the YW tool. The latter was originally developed to allow researchers a simple way to annotate their program scripts in order to reveal the workflow steps and dataflow dependencies implicit in those scripts. With or2yw the user can automatically generate YW models from OpenRefine operation histories, thus providing a 'workflow view' on a previously executed sequence of data cleaning operations. The or2yw tool can generate different types of YesWorkflow models, e.g., a linear model which mirrors the sequential execution order of operations in OpenRefine, and a \emph{parallel model} which reveals independent workflow branches, based on a simple analysis of dependencies between steps: if two operations are independent of each other (e.g., when the columns they read and write do not overlap) then these can be viewed as parallel steps in the data cleaning workflow. The resulting YW models can be understood as a form of prospective provenance, i.e., knowledge artifacts that can be queried and visualized (i) to help authors document their own data cleaning workflows, thereby increasing transparency, and (ii) to help other users, who might want to reuse such workflows, to understand them better.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑