arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1912.08397cs.DC
收藏

动态流工作流的自适应调度

Adaptive Scheduling for Efficient Execution of Dynamic Stream Workflows

Mutaz Barika, Saurabh Garg, Rajiv Ranjan

摘要:

在线异常检测或在线流量监控等流工作流应用将多个流大数据应用集成到数据分析管道中。这个应用程序在本质上是高度动态的,其中数据速度可能在运行时改变,因此资源应该随时间进行管理。为了管理这些变化,这个应用程序的编排需要一个动态的执行环境和动态调度技术。对于前一个需求,多云环境是一个可见的解决方案,以应对这个工作流应用程序的动态方面。而对于后一种需求,动态调度技术不仅需要满足最终用户在数据处理和决策截止时间方面的需求,以及数据流源位置约束,而且需要在运行时调整供应和调度计划,以应对流数据率的动态变化。因此,我们提出了一个两阶段的自适应调度技术,以有效地调度在多云环境中的动态工作流应用程序,可以响应在运行时的数据速度的变化。实验结果表明,该方法接近下限,对不同的实验场景是有效的。

英文摘要:

Stream workflow application such as online anomaly detection or online traffic monitoring, integrates multiple streaming big data applications into data analysis pipeline. This application can be highly dynamic in nature, where the data velocity may change at runtime and therefore the resources should be managed overtime. To manage these changes, the orchestration of this application requires a dynamic execution environment and dynamic scheduling technique. For the former requirement, Multicloud environment is a visible solution to cope with the dynamic aspects of this workflow application. While for the latter requirement, dynamic scheduling technique not only need to adhere to end user's requirements in terms of data processing and deadline for decision making, and data stream sources location constraints, but also adjust provisioning and scheduling plan at runtime to cope with dynamic variations of stream data rates. Therefore, we propose a two-phase adaptive scheduling technique to efficiently schedule dynamic workflow application in Multicloud environment that can respond to changes in the velocity of data at runtime. The experimental results showed that the proposed technique is close to the lower bound and effective for different experiment scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑