arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2407.15363cs.DB

为云端绘制蓝图:使用BRAD统一并自动优化云数据基础设施——扩展版

Blueprinting the Cloud: Unifying and Automatically Optimizing Cloud Data Infrastructures with BRAD -- Extended Version

Geoffrey X. Yu, Ziniu Wu, Ferdi Kossmann, Tianyu Li, Markos Markakis, Amadou Ngom, Samuel Madden, Tim Kraska

AI总结:

本文提出蓝图规划技术与首个云数据虚拟化系统BRAD,通过统一设计空间和基于成本的优化自动选择查询引擎、管理资源并适应工作负载变化,相比无服务器自动扩展或HTAP系统将成本节约提升1.6至13倍。

摘要:

现代组织使用各种专门的云数据库引擎(如Aurora、BigQuery等)来管理数据。然而,设计和维护此类基础设施十分困难。开发者必须考虑许多可能存在不明显性能后果的设计方案;此外,当前的软件抽象将应用程序与特定系统紧密耦合(例如,使用特定引擎的客户端),使得在初始部署后难以更改。更好的解决方案是虚拟化云数据管理,允许开发者声明式地指定其工作负载需求,并依赖自动化解决方案来设计和管理物理实现。在本文中,我们提出了一种称为蓝图规划的技术来实现这一愿景。关键思想是将数据基础设施设计决策投影到统一的设计空间(蓝图)中。然后,我们使用基于成本的优化对候选蓝图进行系统搜索,并利用学习模型来预测蓝图在工作负载上的效用。我们使用该技术构建了BRAD,这是首个云数据虚拟化系统。BRAD用户向单个SQL接口发出查询,该接口可由多个云数据库服务支持。BRAD自动为每个查询选择最合适的引擎,配置和管理资源以最小化成本,并演进基础设施以适应工作负载的转变。我们的评估表明,与无服务器自动扩展或HTAP系统相比,BRAD满足用户定义的性能目标,并将成本节约提高了1.6至13倍。

英文摘要:

Modern organizations manage their data with a wide variety of specialized cloud database engines (e.g., Aurora, BigQuery, etc.). However, designing and managing such infrastructures is hard. Developers must consider many possible designs with non-obvious performance consequences; moreover, current software abstractions tightly couple applications to specific systems (e.g., with engine-specific clients), making it difficult to change after initial deployment. A better solution would virtualize cloud data management, allowing developers to declaratively specify their workload requirements and rely on automated solutions to design and manage the physical realization. In this paper, we present a technique called blueprint planning that achieves this vision. The key idea is to project data infrastructure design decisions into a unified design space (blueprints). We then systematically search over candidate blueprints using cost-based optimization, leveraging learned models to predict the utility of a blueprint on the workload. We use this technique to build BRAD, the first cloud data virtualization system. BRAD users issue queries to a single SQL interface that can be backed by multiple cloud database services. BRAD automatically selects the most suitable engine for each query, provisions and manages resources to minimize costs, and evolves the infrastructure to adapt to workload shifts. Our evaluation shows that BRAD meet user-defined performance targets and improve cost-savings by 1.6-13x compared to serverless auto-scaling or HTAP systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑