arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2105.00129cs.DCcs.SE
收藏

WfChef:自动生成精确的科学工作流生成器

WfChef: Automated Generation of Accurate Scientific Workflow Generators

Tainã Coleman, Henri Casanova, Rafael Ferreira da Silva

摘要:

科学工作流应用已经成为主流,它们在大规模计算平台上的自动化和高效执行是广泛研究和开发的对象。为了使这些努力取得成功,需要一个坚实的实验方法来评估工作流算法和系统。这种方法的基础是现实的工作流实例的可用性。在公共存储库中有几十个用于一些科学应用程序的工作流实例。虽然这些都是非常宝贵的,但它们是有限的:工作流实例并不适用于所有感兴趣的应用程序规模。为了解决这一限制,以前的工作已经开发出合成的,但代表性的,任意规模的工作流实例的生成器。这些生成器很受欢迎,但实现它们是一个手动的劳动密集型过程,需要专业的应用程序知识。因此,这些生成器只针对少数应用程序,尽管数百个应用程序在生产中使用工作流。在这项工作中,我们提出了WfChef,一个框架,完全自动化的过程中构建一个合成的工作流生成器的任何科学应用。基于一组输入的工作流实例,WfChef自动生成一个合成工作流生成器。我们定义和评估几个指标,量化生成的工作流的现实主义。使用这些指标,我们将WfChef生成器生成的工作流的真实性与以前可用的手工生成器生成的工作流进行比较。我们发现,WfChef生成器不仅需要零开发工作(因为它是自动生成的),而且生成的工作流比手工生成器生成的工作流更逼真。

英文摘要:

Scientific workflow applications have become mainstream and their automated and efficient execution on large-scale compute platforms is the object of extensive research and development. For these efforts to be successful, a solid experimental methodology is needed to evaluate workflow algorithms and systems. A foundation for this methodology is the availability of realistic workflow instances. Dozens of workflow instances for a few scientific applications are available in public repositories. While these are invaluable, they are limited: workflow instances are not available for all application scales of interest. To address this limitation, previous work has developed generators of synthetic, but representative, workflow instances of arbitrary scales. These generators are popular, but implementing them is a manual, labor-intensive process that requires expert application knowledge. As a result, these generators only target a handful of applications, even though hundreds of applications use workflows in production. In this work, we present WfChef, a framework that fully automates the process of constructing a synthetic workflow generator for any scientific application. Based on an input set of workflow instances, WfChef automatically produces a synthetic workflow generator. We define and evaluate several metrics for quantifying the realism of the generated workflows. Using these metrics, we compare the realism of the workflows generated by WfChef generators to that of the workflows generated by the previously available, hand-crafted generators. We find that the WfChef generators not only require zero development effort (because it is automatically produced), but also generate workflows that are more realistic than those generated by hand-crafted generators.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑