Skyrise:利用无服务器云基础设施进行弹性数据处理
Skyrise: Exploiting Serverless Cloud Infrastructure for Elastic Data Processing
AI总结:
本文提出了首个完全基于无服务器基础设施的SQL查询处理器Skyrise,通过自适应和成本感知技术缓解无服务器计算在长时任务中的性能与成本局限,在TPC-H基准TB级查询中表现优异。
摘要:
无服务器计算提供了传统基于服务器的云基础设施无法比拟的弹性。尽管现代数据处理系统采用了无服务器存储(如 Amazon S3),但它们仍将计算资源作为服务器进行管理。这对于不可预测的工作负载而言极具挑战性,常导致集群利用率低下。近期研究表明,云函数等无服务器计算资源在弹性数据处理方面具有潜力,但在长时间运行的工作负载中存在性能鲁棒性和成本效率的局限。这些挑战需要在整个系统堆栈中采用全局性方法。然而,据我们所知,目前尚无完全构建于无服务器基础设施之上的端到端数据处理系统。本文提出了 Skyrise,旨在构建首个完全无服务器的 SQL 查询处理器。Skyrise 利用底层基础设施的弹性,同时通过多种自适应和成本感知技术缓解了固有限制。实验表明,在分析型 TPC-H 基准的 TB 级查询中,Skyrise 的性能和成本均可与其他云数据系统相媲美。
英文摘要:
Serverless computing offers elasticity unmatched by conventional server-based cloud infrastructure. Although modern data processing systems embrace serverless storage, such as Amazon S3, they continue to manage their compute resources as servers. This is challenging for unpredictable workloads, leaving clusters often underutilized. Recent research shows the potential of serverless compute resources, such as cloud functions, for elastic data processing, but also sees limitations in performance robustness and cost efficiency for long running workloads. These challenges require holistic approaches across the system stack. However, to the best of our knowledge, there is no end-to-end data processing system built entirely on serverless infrastructure. In this paper, we present Skyrise, our effort towards building the first fully serverless SQL query processor. Skyrise exploits the elasticity of its underlying infrastructure, while alleviating the inherent limitations with a number of adaptive and cost-aware techniques. We show that both Skyrise's performance and cost are competitive to other cloud data systems for terabyte-scale queries of the analytical TPC-H benchmark.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。