基于P2P的志愿计算工作流的自适应检查点机制
An Adaptive Checkpointing Scheme for Peer-to-Peer Based Volunteer Computing Work Flows
摘要:
志愿者计算(Volunteer Computing),有时也称为公共资源计算(Public Resource Computing),是一种新兴的计算模型,非常适合工作池并行处理。随着越来越复杂的网格应用程序在设计和部署中使用工作流,考虑工作流部署对志愿者计算基础设施的影响是合理的。在这种情况下,工作流间I/O会导致工作池服务器上的I/O需求显著增加。一个可能的解决方案是使用基于对等的并行计算架构来卸载这个I/O需求的工人,其中的工人可以履行工作流协调和I/O检查等的某些方面,但是,在这样一个大规模的系统实现鲁棒性是一个具有挑战性的障碍,对分散执行的工作流和一般的并行处理。为了提高鲁棒性,我们提出并展示了使用自适应检查点方案的优点,该方案根据相关网络和对等参数的估计有效地检查点并行进程的状态。我们的方案使用在运行时观察到的统计数据,以完全分散的方式动态地做出检查点决策。仿真结果表明,支持我们提出的方法在减少所需的运行时间。
英文摘要:
Volunteer Computing, sometimes called Public Resource Computing, is an emerging computational model that is very suitable for work-pooled parallel processing. As more complex grid applications make use of work flows in their design and deployment it is reasonable to consider the impact of work flow deployment over a Volunteer Computing infrastructure. In this case, the inter work flow I/O can lead to a significant increase in I/O demands at the work pool server. A possible solution is the use of a Peer-to- Peer based parallel computing architecture to off-load this I/O demand to the workers; where the workers can fulfill some aspects of work flow coordination and I/O checking, etc. However, achieving robustness in such a large scale system is a challenging hurdle towards the decentralized execution of work flows and general parallel processes. To increase robustness, we propose and show the merits of using an adaptive checkpoint scheme that efficiently checkpoints the status of the parallel processes according to the estimation of relevant network and peer parameters. Our scheme uses statistical data observed during runtime to dynamically make checkpoint decisions in a completely de- centralized manner. The results of simulation show support for our proposed approach in terms of reduced required runtime.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。