CheckMate：评估流数据流的检查点协议
CheckMate: Evaluating Checkpointing Protocols for Streaming Dataflows
AI总结：
本文首次对比评估了流数据流中协调、非协调和通信诱导三种检查点协议，发现非协调方法在倾斜工作负载下优于协调方法，建议未来研究重点优化非协调检查点机制。
摘要：
在过去十年中，流处理在商业和研究环境中得到了广泛采用。这一成功的一个关键因素是现代流处理器在确保精确一次处理保证的同时处理故障的能力。在撰写本文时，几乎所有保证精确一次处理的流处理器都实现了 Apache Flink 的协调检查点的变体——这是对 1985 年原始 Chandy-Lamport 检查点的扩展。然而，正如流处理社区的从业者所报告的，这种协调方法之所以如此普遍的原因仍然是轶事。同时，常见的检查点方法，如非协调和通信诱导的方法，在很大程度上仍未被探索。本文首次解决了这一差距，通过 i）阐明从业者为何偏爱协调方法，以及 ii）调查是否存在可行的替代方案。为此，我们实现了三种检查点方法，这些方法是我们调查并适应流数据流独特需求而得到的。我们的分析表明，在均匀分布的工作负载下，协调方法优于非协调和通信诱导协议。然而，令我们惊讶的是，非协调方法不仅在均匀分布的工作负载中与协调方法具有竞争力，而且在倾斜工作负载中优于协调方法。我们得出结论，与其盲目地采用协调检查点，研究应集中于优化非常有前途的非协调方法，因为它可以解决倾斜问题并支持流行的循环查询。我们相信我们的发现可以引发对检查点机制的进一步研究。
英文摘要：
Stream processing in the last decade has seen broad adoption in both commercial and research settings. One key element for this success is the ability of modern stream processors to handle failures while ensuring exactly-once processing guarantees. At the moment of writing, virtually all stream processors that guarantee exactly-once processing implement a variant of Apache Flink's coordinated checkpoints - an extension of the original Chandy-Lamport checkpoints from 1985. However, the reasons behind this prevalence of the coordinated approach remain anecdotal, as reported by practitioners of the stream processing community. At the same time, common checkpointing approaches, such as the uncoordinated and the communication-induced ones, remain largely unexplored. This paper is the first to address this gap by i) shedding light on why practitioners have favored the coordinated approach and ii) by investigating whether there are viable alternatives. To this end, we implement three checkpointing approaches that we surveyed and adapted for the distinct needs of streaming dataflows. Our analysis shows that the coordinated approach outperforms the uncoordinated and communication-induced protocols under uniformly distributed workloads. To our surprise, however, the uncoordinated approach is not only competitive to the coordinated one in uniformly distributed workloads, but it also outperforms the coordinated approach in skewed workloads. We conclude that rather than blindly employing coordinated checkpointing, research should focus on optimizing the very promising uncoordinated approach, as it can address issues with skew and support prevalent cyclic queries. We believe that our findings can trigger further research into checkpointing mechanisms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
