使用TPCx-HS评估Hadoop集群
Evaluating Hadoop Clusters with TPCx-HS
摘要：
大数据平台的复杂性和多样性不断增加，使得所有系统用户正确设置和操作系统变得既困难又耗时。另一个挑战是比较平台，以便为特定应用选择最合适的平台。所有这些因素都促使人们需要一个标准化的大数据基准，以帮助用户进行平台评估。就在最近，TPCx-HS [1][2]作为第一个标准化的大数据基准被发布，旨在对Hadoop集群进行压力测试。本研究的目的是评估和比较网络设置如何影响Hadoop集群的性能。特别是，实验是使用由相同的Cloudera Hadoop Distribution（CDH）集群设置使用的共享和专用1Gbit网络进行的。TPCx-HS基准测试是非常网络密集型的，用于压力测试和比较两种集群设置。所有呈现的结果都是通过使用官方可用的基准版本[1]获得的，但它们与官方报告的结果不具有可比性，并且是作为实验评估，未经任何外部组织审核。正如预期的那样，专用1Gbit网络设置的执行速度比共享1Gbit设置快得多。然而，令人惊讶的是，这两种集群设置之间的价格差异可以忽略不计，这带来了成倍的性能回报。
英文摘要：
The growing complexity and variety of Big Data platforms makes it both difficult and time consuming for all system users to properly setup and operate the systems. Another challenge is to compare the platforms in order to choose the most appropriate one for a particular application. All these factors motivate the need for a standardized Big Data benchmark that can help the users in the process of platform evaluation. Just recently TPCx-HS [1][2] has been released as the first standardized Big Data benchmark designed to stress test a Hadoop cluster. The goal of this study is to evaluate and compare how the network setup influences the performance of a Hadoop cluster. In particular, experiments were performed using shared and dedicated 1Gbit networks utilized by the same Cloudera Hadoop Distribution (CDH) cluster setup. The TPCx-HS benchmark, which is very network intensive, was used to stress test and compare both cluster setups. All the presented results are obtained by using the officially available version [1] of the benchmark, but they are not comparable with the officially reported results and are meant as an experimental evaluation, not audited by any external organization. As expected the dedicated 1Gbit network setup performed much faster than the shared 1Gbit setup. However, what was surprising is the negligible price difference between both cluster setups, which pays off with a multifold performance return.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
