arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1810.04599cs.DB
收藏

通过图分割和总结理解数据科学的起源

Understanding Data Science Lifecycle Provenance via Graph Segmentation and Summarization

Hui Miao, Amol Deshpande

摘要:

如今越来越现代的数据科学平台具有非侵入性和可扩展的起源摄取机制,以收集丰富的起源和上下文信息,使用可区分的版本来处理对同一文件的修改,并使用图形数据模型(例如,属性图)和查询语言(例如,Cypher)来表示和操作存储的出处/上下文信息。由于元数据的模式后期性质、相同文件的多个版本以及团队成员引入的不熟悉的工件,“起源图”是冗长的和不断发展的,并且难以理解;使用标准的图查询模型,很难组合查询并利用这些有价值的信息。在本文中,我们提出了两个高层次的图查询运算符,以解决这种起源图的冗长性和不断发展的性质。首先,我们引入了一个图分割算子,它通过灵活的边界条件查询一组源顶点和一组目标顶点之间的追溯起源,以帮助用户了解这些顶点之间的派生关系。我们展示了这样一个查询的语义方面的上下文无关文法,并开发高效的算法,运行速度比国家的最先进的数量级。第二,我们提出了一个图形汇总操作,结合类似的部分在一起查询潜在的出处的基础项目。该操作允许通过忽略顶点细节和表征局部结构来调整摘要,并使用路径约束来确保起源意义。我们证明了最优摘要问题是PSPACE完备的,并提出了有效的近似算法。这些操作符是在属性图后端之上实现的。我们广泛评估我们的查询方法,并显示所提出的方法的有效性和效率。

英文摘要:

Increasingly modern data science platforms today have non-intrusive and extensible provenance ingestion mechanisms to collect rich provenance and context information, handle modifications to the same file using distinguishable versions, and use graph data models (e.g., property graphs) and query languages (e.g., Cypher) to represent and manipulate the stored provenance/context information. Due to the schema-later nature of the metadata, multiple versions of the same files, and unfamiliar artifacts introduced by team members, the "provenance graph" is verbose and evolving, and hard to understand; using standard graph query model, it is difficult to compose queries and utilize this valuable information. In this paper, we propose two high-level graph query operators to address the verboseness and evolving nature of such provenance graphs. First, we introduce a graph segmentation operator, which queries the retrospective provenance between a set of source vertices and a set of destination vertices via flexible boundary criteria to help users get insight about the derivation relationships among those vertices. We show the semantics of such a query in terms of a context-free grammar, and develop efficient algorithms that run orders of magnitude faster than state-of-the-art. Second, we propose a graph summarization operator that combines similar segments together to query prospective provenance of the underlying project. The operator allows tuning the summary by ignoring vertex details and characterizing local structures, and ensures the provenance meaning using path constraints. We show the optimal summary problem is PSPACE-complete and develop effective approximation algorithms. The operators are implemented on top of a property graph backend. We evaluate our query methods extensively and show the effectiveness and efficiency of the proposed methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑