arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2111.12420cs.PL
收藏

CircuitFlow:一种用于数据流编程的领域特定语言(附附录)

CircuitFlow: A Domain Specific Language for Dataflow Programming (with appendices)

Riley Evans, Samantha Frohlich, Meng Wang

摘要:

数据流应用程序(如机器学习算法)可以运行数天,因此需要确保它们能够正确工作。目前的工具还不够好:任务之间的交互往往不是类型安全的,导致不希望的运行时错误。本文提出了一种新的声明式Haskell嵌入式DSL(eDSL)的底层编程:CircuitFlow。定义为对工作流中的依赖关系进行建模的数据上的对称Monoidal前序(SMP),它具有强大的数学基础,重新关注数据流如何通过应用程序,从而产生更具表现力的解决方案,不仅静态捕获错误,而且还实现具有竞争力的运行时性能。在我们的初步评估中,CircuitFlow通过更好地扩展输入数量而优于业界领先的Spotify Luigi库。CircuitFlow的创新性创建也值得注意,它举例说明了如何创建一个模块化的eDSL,其语义需要效果,并且存储程序正确性的复杂类型信息至关重要。

英文摘要:

Dataflow applications, such as machine learning algorithms, can run for days, making it desirable to have assurances that they will work correctly. Current tools are not good enough: too often the interactions between tasks are not type-safe, leading to undesirable run-time errors. This paper presents a new declarative Haskell Embedded DSL (eDSL) for dataflow programming: CircuitFlow. Defined as a Symmetric Monoidal Preorder (SMP) on data that models dependencies in the workflow, it has a strong mathematical basis, refocusing on how data flows through an application, resulting in a more expressive solution that not only catches errors statically, but also achieves competitive run-time performance. In our preliminary evaluation, CircuitFlow outperforms the industry-leading Luigi library of Spotify by scaling better with the number of inputs. The innovative creation of CircuitFlow is also of note, exemplifying how to create a modular eDSL whose semantics necessitates effects, and where storing complex type information for program correctness is paramount.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑