状态流的完整融合:状态流的等式理论与融合作为规范化-评估
Complete Fusion for Stateful Streams: Equational Theory of Stateful Streams and Fusion as Normalization-by-Evaluation
AI总结:
Strymonas通过等式理论实现状态流的完整融合,提供高性能且可移植的流处理库,支持复杂管道构建与状态管理。
摘要:
处理大量数据快速且在常数和小空间内是流处理的关键点,也是其日益普及的原因。然而,最高效的、命令式处理代码几乎难以阅读,更不用说修改、重用或正确编写了。我们提出了流编译理论及其实现,即一个可移植的流处理库Strymonas,使我们能够通过简单组合子的拼接来构建复杂的流管道,同时达到手写命令式循环和状态机的性能。该库支持有限和无限流,并提供丰富的组合子。它们可以自由组合,但最终的命令式代码没有任何组合子抽象的痕迹:没有闭包或中间对象。高性能是可移植且静态保证的,不依赖于编译器或黑箱优化。我们大大超过了OCaml中可用的流处理库的性能。该库生成C和OCaml代码。声明性构建的Strymonas管道都是状态性的。库中引入的状态不可直接观察。因此,Strymonas的API看起来像熟悉的`纯函数式`组合子接口。程序员可以引入自己的流状态并在管道中共享。Strymonas与状态流的等式理论同时开发。我们的理论模型代表了所有期望的管道,并保证了唯一正常形式的存在,这些正常形式可以映射到(融合)状态机。我们描述了规范化算法,作为规范化-评估的一种形式。等式理论使我们能够陈述并证明完整融合优化的正确性。
英文摘要:
Processing large amounts of data fast, in constant and small space is the point of stream processing and the reason for its increasing use. Alas, the most performant, imperative processing code tends to be almost impossible to read, let alone modify, reuse -- or write correctly. We present both a stream compilation theory and its implementation as a portable stream processing library Strymonas that lets us assemble complex stream pipelines just by plugging in simple combinators, and yet attain the performance of hand-written imperative loops and state machines. The library supports finite and infinite streams and offers a rich set of combinators. They may be freely composed, and yet the resulting convoluted imperative code has no traces of combinator abstractions: no closures or intermediate objects. The high-performance is portable and statically guaranteed, without relying on compiler or black-box optimizations. We greatly exceed in performance the available stream processing libraries in OCaml. The library generates C and OCaml code. The declaratively built Strymonas pipelines are all stateful. The stream state introduced in the library is not directly observable. Therefore, the Strymonas API looks like the familiar interface of `pure functional' combinators. Programmers may introduce their own stream state and share it across the pipeline. Strymonas has been developed in tandem with the equational theory of stateful streams. Our theoretical model represents all desired pipelines and guarantees the existence of unique normal forms, which are mappable to (fused) state machines. We describe the normalization algorithm, as a form of normalization-by-evaluation. The equational theory lets us state and prove the correctness of the complete fusion optimization.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。