arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1903.09347cs.OScs.PF
收藏

了解和驯服SSD读取性能可变性:HDFS案例研究

Understanding and taming SSD read performance variability: HDFS case study

María F. Borge, Florin Dinu, Willy Zwaenepoel

摘要:

本文分析了低层(文件系统、操作系统、SSD)对HDFS在读路径上从SSD中提取最大性能的影响,发现并分析了低层导致HDFS读吞吐量损失的三种令人惊讶的性能下降。首先,固有的速度降低会在一段可变的时间内影响从每个新文件系统扩展区的读取。第二,时间减速是暂时的和周期性的,并且是工作负载不可知的。第三,在永久减速中,一些文件在一段时间后可能会单独地永久地变慢。我们分析了这些速度减慢对HDFS的影响,并显示出显着的吞吐量损失。单独地,每一次减速都可能导致10- 15%的读取吞吐量损失。然而,其影响是累积的。当所有的减速同时发生时,读取吞吐量下降多达30%。我们进一步分析缓解技术,并表明,三个减速中的两个可以通过在较低层中增加IO请求并行性来解决。不幸的是,HDFS不能自动适应使用这种额外的并行性。我们的研究结果表明,需要在存储堆栈的适应性。原因在于,在通常情况下使性能最大化的访问模式不一定是可以掩盖性能波动的相同访问模式。

英文摘要:

In this paper we analyze the influence that lower layers (file system, OS, SSD) have on HDFS' ability to extract maximum performance from SSDs on the read path. We uncover and analyze three surprising performance slowdowns induced by lower layers that result in HDFS read throughput loss. First, intrinsic slowdown affects reads from every new file system extent for a variable amount of time. Second, temporal slowdown appears temporarily and periodically and is workload-agnostic. Third, in permanent slowdown, some files can individually and permanently become slower after a period of time. We analyze the impact of these slowdowns on HDFS and show significant throughput loss. Individually, each of the slowdowns can cause a read throughput loss of 10-15%. However, their effect is cumulative. When all slowdowns happen concurrently, read throughput drops by as much as 30%. We further analyze mitigation techniques and show that two of the three slowdowns could be addressed via increased IO request parallelism in the lower layers. Unfortunately, HDFS cannot automatically adapt to use such additional parallelism. Our results point to a need for adaptability in storage stacks. The reason is that an access pattern that maximizes performance in the common case is not necessarily the same one that can mask performance fluctuations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑