arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1607.04334cs.DC
收藏

面向随机优化数据计算流

Towards Stochastically Optimizing Data Computing Flows

Farshid Farhat, Diman Zad Tootaghaj, Mohammad Arjomand

摘要:

随着内存密集型应用程序产生的非结构化数据量的快速增长,大规模数据分析最近引起了越来越多的兴趣。处理、管理和分析这些海量数据给云和数据中心计算领域带来了一些挑战。特别地,用于分布式数据分析的传统框架基于不同数据处理节点的同质性和非随机分布的假设。本文讨论了扩展大数据计算的基本限制因素。结果表明,随着串行和并行计算服务器的数量增加,尾部(平均值和方差)的作业执行时间增加。我们将首先提出一个模型来预测高度分布式处理任务的响应时间,然后提出一个新的实用计算算法来优化响应时间。

英文摘要:

With rapid growth in the amount of unstructured data produced by memory-intensive applications, large scale data analytics has recently attracted increasing interest. Processing, managing and analyzing this huge amount of data poses several challenges in cloud and data center computing domain. Especially, conventional frameworks for distributed data analytics are based on the assumption of homogeneity and non-stochastic distribution of different data-processing nodes. The paper argues the fundamental limiting factors for scaling big data computation. It is shown that as the number of series and parallel computing servers increase, the tail (mean and variance) of the job execution time increase. We will first propose a model to predict the response time of highly distributed processing tasks and then propose a new practical computational algorithm to optimize the response time.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑