ShuffleBench：一个用于评估分布式流处理框架大规模数据Shuffle操作的基准测试程序
ShuffleBench: A Benchmark for Large-Scale Data Shuffling Operations with Distributed Stream Processing Frameworks
AI总结：
本文提出新型基准测试程序ShuffleBench，用于评估分布式流处理框架的数据shuffle性能，它高度可配置、开源且支持四款主流框架，实验显示Flink吞吐量最高、Hazelcast延迟最低。
摘要：
分布式流处理框架可助力构建可扩展、高可靠的应用，以对连续数据流执行转换和聚合操作。本文提出了ShuffleBench，这是一款用于评估现代流处理框架性能的新型基准测试程序。与其他基准测试不同，它聚焦于这类用例：流处理框架主要用于对数据记录执行shuffle（即重新分发）以完成状态本地聚合，而实际的聚合逻辑被视为黑盒软件组件。ShuffleBench的设计灵感源自某大型云可观测性平台的近实时分析需求，采用了性能工程研究领域已确立的延迟、吞吐量和可扩展性相关基准测试指标与方法。尽管其灵感来自真实的可观测性用例，但它具备高度可配置性，可支持跨领域的性能评估。ShuffleBench是一款开箱即用的开源软件，依托现有Kubernetes工具构建，并提供了四款主流前沿框架的实现。因此，我们认为ShuffleBench既能为构建流处理应用的工业界从业者提供价值，也能为研究新型流处理方案的科研人员提供助力。本文还补充了一项实验性能评估：在云原生环境中，通过多种配置的ShuffleBench对Flink、Hazelcast、Kafka Streams和Spark进行测试。结果显示，Flink实现了最高的吞吐量，而Hazelcast处理数据流的延迟最低。
英文摘要：
Distributed stream processing frameworks help building scalable and reliable applications that perform transformations and aggregations on continuous data streams. This paper introduces ShuffleBench, a novel benchmark to evaluate the performance of modern stream processing frameworks. In contrast to other benchmarks, it focuses on use cases where stream processing frameworks are mainly employed for shuffling (i.e., re-distributing) data records to perform state-local aggregations, while the actual aggregation logic is considered as black-box software components. ShuffleBench is inspired by requirements for near real-time analytics of a large cloud observability platform and takes up benchmarking metrics and methods for latency, throughput, and scalability established in the performance engineering research community. Although inspired by a real-world observability use case, it is highly configurable to allow domain-independent evaluations. ShuffleBench comes as a ready-to-use open-source software utilizing existing Kubernetes tooling and providing implementations for four state-of-the-art frameworks. Therefore, we expect ShuffleBench to be a valuable contribution to both industrial practitioners building stream processing applications and researchers working on new stream processing approaches. We complement this paper with an experimental performance evaluation that employs ShuffleBench with various configurations on Flink, Hazelcast, Kafka Streams, and Spark in a cloud-native environment. Our results show that Flink achieves the highest throughput while Hazelcast processes data streams with the lowest latency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
