arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1408.2284cs.DC
收藏

低功耗处理器中的Hadoop

Hadoop in Low-Power Processors

Da Zheng, Alexander Szalay, Andreas Terzis

摘要:

在我们之前的工作中,我们介绍了一个所谓的Amdahl刀片微服务器,它结合了低功耗的Atom处理器,GPU和SSD,以提供一个平衡和节能的系统。我们的初步结果表明,顺序I/O的Amdahl刀片可以是十倍以上的传统服务器集群具有可比的功耗。在本文中,我们调查的性能和能源效率的Amdahl刀片运行Hadoop。我们的研究结果表明,对于数据密集型和计算密集型应用程序,Amdahl刀片式服务器的能效分别是Open Cloud Consortium集群的7.7倍和3.4倍。Hadoop分布式文件系统在Amdahl刀片上的性能相对较差,因为磁盘和网络I/O在Atom处理器上都是CPU密集型操作。我们展示了三种有效的技术,以减少CPU消耗和提高性能。然而,即使有了这些改进,Atom处理器仍然是系统的瓶颈。我们重新审视了Amdahl定律,并估计Amdahl刀片需要四个Atom内核才能很好地平衡Hadoop任务。

英文摘要:

In our previous work we introduced a so-called Amdahl blade microserver that combines a low-power Atom processor, with a GPU and an SSD to provide a balanced and energy-efficient system. Our preliminary results suggested that the sequential I/O of Amdahl blades can be ten times higher than that a cluster of conventional servers with comparable power consumption. In this paper we investigate the performance and energy efficiency of Amdahl blades running Hadoop. Our results show that Amdahl blades are 7.7 times and 3.4 times as energy-efficient as the Open Cloud Consortium cluster for a data-intensive and a compute-intensive application, respectively. The Hadoop Distributed Filesystem has relatively poor performance on Amdahl blades because both disk and network I/O are CPU-heavy operations on Atom processors. We demonstrate three effective techniques to reduce CPU consumption and improve performance. However, even with these improvements, the Atom processor is still the system's bottleneck. We revisit Amdahl's law, and estimate that Amdahl blades need four Atom cores to be well balanced for Hadoop tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑