给猪涂口红:启用数据库风格的工作流出处
Putting Lipstick on Pig: Enabling Database-style Workflow Provenance
摘要:
工作流起源通常假设每个模块都是一个“黑盒”,因此每个输出都依赖于所有输入(粗粒度依赖)。此外,它不对模块的内部状态进行建模,而模块的内部状态可能会在重复执行之间发生变化。然而,在实践中,输出可能只依赖于输入的一个小子集(细粒度依赖)以及模块的内部状态。我们提出了一种新颖的起源框架,它结合了数据库风格和工作流风格的起源,通过使用Pig Latin来暴露模块的功能,从而捕获内部状态和细粒度的依赖关系。我们的解决方案中的一个关键因素是使用一种新形式的起源图,模型模块调用,并产生一个紧凑的细粒度的工作流起源表示。它还支持许多新颖的图形转换操作,允许在起源查询(放大和缩小)中选择所需的粒度级别,并支持“假设”工作流分析查询。我们在Lipstick系统中实现了我们的方法,并开发了一个支持系统性能评估的基准。我们的研究结果表明,跟踪和查询细粒度的工作流起源的可行性。
英文摘要:
Workflow provenance typically assumes that each module is a "black-box", so that each output depends on all inputs (coarse-grained dependencies). Furthermore, it does not model the internal state of a module, which can change between repeated executions. In practice, however, an output may depend on only a small subset of the inputs (fine-grained dependencies) as well as on the internal state of the module. We present a novel provenance framework that marries database-style and workflow-style provenance, by using Pig Latin to expose the functionality of modules, thus capturing internal state and fine-grained dependencies. A critical ingredient in our solution is the use of a novel form of provenance graph that models module invocations and yields a compact representation of fine-grained workflow provenance. It also enables a number of novel graph transformation operations, allowing to choose the desired level of granularity in provenance querying (ZoomIn and ZoomOut), and supporting "what-if" workflow analytic queries. We implemented our approach in the Lipstick system and developed a benchmark in support of a systematic performance evaluation. Our results demonstrate the feasibility of tracking and querying fine-grained workflow provenance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。