分层数据流模型上的大数据框架比较
A Comparison of Big Data Frameworks on a Layered Dataflow Model
摘要：
在大数据分析领域，有一系列工具旨在简化在集群上执行的应用程序编程。虽然每个工具都声称提供更好的编程，数据和执行模型，但通常只提供非正式的（并且经常令人困惑的）语义，所有这些都共享一个共同的底层模型，即数据流模型。我们提出的数据流模型展示了各种工具如何在不同的抽象层次上共享相同的表达能力。这项工作的贡献是双重的：首先，我们证明了所提出的模型（至少）与现有的批处理和流框架一样通用（例如，Spark，Flink，Storm），从而更容易理解在这些框架中编写的高级数据处理应用程序。其次，我们提供了一个分层的模型，可以表示工具和应用程序的数据流范式，我们展示了如何分析的工具适合在每个级别。
英文摘要：
In the world of Big Data analytics, there is a series of tools aiming at simplifying programming applications to be executed on clusters. Although each tool claims to provide better programming, data and execution models, for which only informal (and often confusing) semantics is generally provided, all share a common underlying model, namely, the Dataflow model. The Dataflow model we propose shows how various tools share the same expressiveness at different levels of abstraction. The contribution of this work is twofold: first, we show that the proposed model is (at least) as general as existing batch and streaming frameworks (e.g., Spark, Flink, Storm), thus making it easier to understand high-level data-processing applications written in such frameworks. Second, we provide a layered model that can represent tools and applications following the Dataflow paradigm and we show how the analyzed tools fit in each level.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
