arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.07771cs.DB
收藏

面向大规模数据处理的无服务器云基础设施实证评估

An Empirical Evaluation of Serverless Cloud Infrastructure for Large-Scale Data Processing

Thomas Bodner, Theo Radig, David Justen, Daniel Ritter, Tilmann Rabl

AI总结:

该研究提出Skyrise无服务器评估平台,通过微基准测试与端到端工作负载分析AWS无服务器基础设施的数据处理性能与成本特性,明确性能波动边界与成本盈亏点,为其高效应用提供指导。

摘要:

数据处理系统正越来越多地部署在云端。单体系统完全运行在虚拟服务器上,而近期的系统则依托云基础设施,利用计算与存储的解耦实现二者的独立扩展。无服务器计算服务(如AWS Lambda)的推出,为这些系统带来了更细粒度的弹性扩展能力。已有研究证明了无服务器基础设施可用于可扩展数据处理,但也发现其存在性能波动和成本开销方面的局限,尤其是在网络和存储环节。\n 本文针对数据处理场景下的无服务器基础设施,开展了性能与成本特性的详细分析。我们的分析基于跨不同计算与存储服务的大量微基准测试,以及端到端工作负载。为支撑本次分析,我们提出了Skyrise无服务器评估平台。针对应用广泛的AWS无服务器基础设施,我们的分析揭示了无服务器网络与存储中性能波动的明确边界,还给出了无服务器计算与存储的成本盈亏平衡点。这些发现为无服务器基础设施何时、如何高效应用于数据处理提供了指导。

英文摘要:

Data processing systems are increasingly deployed in the cloud. While monolithic systems run fully on virtual servers, recent systems embrace cloud infrastructure and utilize the disaggregation of compute and storage to scale them independently. The introduction of serverless compute services, such as AWS Lambda, enables finer-grained and elastic scalability within these systems. Prior work shows the viability of serverless infrastructure for scalable data processing yet also sees limitations due to variable performance and cost overhead, in particular for networking and storage. In this paper, we perform a detailed analysis of the performance and cost characteristics of serverless infrastructure in the data processing context. We base our analysis on a large series of micro-benchmarks across different compute and storage services, as well as end-to-end workloads. To enable our analysis, we propose the Skyrise serverless evaluation platform. For the widely used serverless infrastructure of AWS, our analysis reveals distinct boundaries for performance variability in serverless networks and storage. We further present cost break-even points for serverless compute and storage. These insights provide guidance on when and how serverless infrastructure can be efficiently used for data processing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑