arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1806.01109cs.DC
收藏

数据流上复杂事件处理系统的自适应并行处理策略

An adaptive parallel processing strategy in complex event processing systems over data streams

Fuyuan Xiao, Masayoshi Aritsugi

摘要:

数据流事件与持久查询的有效匹配是事件流处理系统的基础。这些应用需要在分布式复杂事件处理(CEP)系统上处理大容量和连续的数据流,并具有快速的处理时间。因此,需要一种管理良好的并行处理技术来提高系统的性能。然而,CEP系统中模式算子的特殊性质增加了并行处理问题的难度。针对这些问题,通过引入直方图,利用概率论和排队论,提出了复杂事件处理的并行化模型和自适应并行处理策略。所提出的策略可以估计最优的事件分裂策略,它可以适合最近的工作负载条件,使所选择的策略具有最少的预期等待时间,以进一步处理到达的事件。该策略能够在操作员时间窗口大小和流输入速率变化的情况下保持CEP系统的快速运行。最后,通过在StreamBase系统上的实验,验证了本文工作的实用性。

英文摘要:

Efficient matching of incoming events of data streams to persistent queries is fundamental to event stream processing systems. These applications require dealing with high volume and continuous data streams with fast processing time on distributed complex event processing (CEP) systems. Therefore, a well-managed parallel processing technique is needed for improving the performance of the system. However, the specific properties of pattern operators in the CEP systems increase the difficulties of the parallel processing problem. To address these issues, a parallelization model and an adaptive parallel processing strategy are proposed for the complex event processing by introducing a histogram, and utilizing the probability and queue theory. The proposed strategy can estimate the optimal event splitting policy, which can suit the most recent workloads conditions such that the selected policy has the least expected waiting time for further processing the arriving events. The proposed strategy can keep the CEP system running fast under the variation of the time window sizes of operators and input rates of streams. Finally, the utility of our work is demonstrated through the experiments on the StreamBase system.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑