arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2203.03501cs.DC
收藏

迁移与不迁移:分布式流处理中操作符迁移的分析

To Migrate or not to Migrate: An Analysis of Operator Migration in Distributed Stream Processing

Espen Volnes, Thomas Plagemann, Vera Goebel

摘要:

数据流处理系统中最重要的问题之一是使用操作员迁移以具有成本效益的方式处理高度可变的工作负载,并根据需要在任何给定时间适应需求。操作符迁移是一个复杂的过程,涉及到正在运行的查询的状态和流管理的更改,通常不会丢失任何数据,并且尽可能少地中断执行。本调查从历史角度以及迁移目标的角度概述了运营商迁移的解决方案。它引入了操作员迁移的概念模型,以建立统一的术语和分类现有的解决方案。现有的工作在该领域进行了分析,迁移的机制从迁移数据的决定。在后一种情况下,强调成本效益分析,这对运营商迁移很重要,但往往只是隐含地解决,或完全被忽视。对可用解决方案的描述使读者从算法的角度对设计方案有了很好的理解。我们补充这一点的实证研究,提供定量的见解不同的设计方案的迁移机制的影响。

英文摘要:

One of the most important issues in data stream processing systems is to use operator migration to handle highly variable workloads in a cost-efficient manner and adapt to the needs at any given time on demand. Operator migration is a complex process that involves changes in the state and stream management of a running query, typically without any loss of data, and with as little disruption to the execution as possible. This survey provides an overview of solutions for operator migration from a historical perspective as well as the perspective of the goal of migration. It introduces a conceptual model of operator migration to establish a unified terminology and classify existing solutions. Existing work in the area is analyzed to separate the mechanism of migration from the decision to migrate the data. In case of the latter, a cost-benefit analysis is emphasized that is important for operator migration but is often only implicitly addressed, or is neglected altogether. A description of the available solutions provides the reader with a good understanding of the design alternatives from an algorithmic viewpoint. We complement this with an empirical study to provide quantitative insights on the impact of different design alternatives on the mechanisms of migration.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑