arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1409.0792cs.PF
收藏

大数据工作负载的特征化和子集化

Characterizing and Subsetting Big Data Workloads

Zhen Jia, Jianfeng Zhan, Lei Wang, Rui Han, Sally A. McKee, Qiang Yang, Chunjie Luo, Jingwei Li

摘要:

大数据基准测试套件必须包括多种数据和工作负载,才能在公平评估大数据系统和架构时发挥作用。然而,使用真正全面的基准对架构社区提出了巨大的挑战。首先,我们需要彻底了解各种工作负载的行为。其次,我们通常基于模拟的研究方法对于大数据来说变得过于昂贵。大数据是一个新兴的领域,越来越多的软件栈被提出来促进大数据应用程序的开发,这加剧了这些挑战。在本文中,我们首先使用主成分分析(PCA)从45个指标中识别出最重要的特征,以表征BigDataBench(一个全面的大数据基准测试套件)中的大数据工作负载。其次,我们将聚类技术应用于从PCA获得的主成分,以调查大数据工作负载之间的相似性,并验证了包括不同软件堆栈对大数据基准测试的重要性。第三,我们通过删除冗余工作负载,选择了七个具有代表性的大数据工作负载,并发布了BigDataBench模拟版本,该版本可从http://prof.ict.ac.cn/BigDataBench/simulatorversion/公开获取。

英文摘要:

Big data benchmark suites must include a diversity of data and workloads to be useful in fairly evaluating big data systems and architectures. However, using truly comprehensive benchmarks poses great challenges for the architecture community. First, we need to thoroughly understand the behaviors of a variety of workloads. Second, our usual simulation-based research methods become prohibitively expensive for big data. As big data is an emerging field, more and more software stacks are being proposed to facilitate the development of big data applications, which aggravates hese challenges. In this paper, we first use Principle Component Analysis (PCA) to identify the most important characteristics from 45 metrics to characterize big data workloads from BigDataBench, a comprehensive big data benchmark suite. Second, we apply a clustering technique to the principle components obtained from the PCA to investigate the similarity among big data workloads, and we verify the importance of including different software stacks for big data benchmarking. Third, we select seven representative big data workloads by removing redundant ones and release the BigDataBench simulation version, which is publicly available from http://prof.ict.ac.cn/BigDataBench/simulatorversion/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑