在Microsoft的生产环境中部署导向查询优化器
Deploying a Steered Query Optimizer in Production at Microsoft
摘要:
现代分析工作负载高度异构且非常复杂,使得通用查询优化器对于许多客户和场景来说都是站不住脚的。因此,将这些优化器专门用于工作负载的实例是很重要的。在本文中,我们继续最近的工作路线,在转向查询优化器更好的计划,为给定的工作负载,并取得重大进展,推动以前的研究思路,生产部署。在此过程中,我们解决了几个运营挑战,包括使转向操作更易于管理,将转向成本保持在预算范围内,以及避免生产中出现意外的性能下降。我们由此产生的系统,QQ顾问,本质上是外化的查询规划器到一个巨大的离线管道,更好的探索和专业化。我们讨论了我们的设计的各个方面,并在Microsoft的生产范围工作负载上显示详细的结果,其中系统当前默认启用。
英文摘要:
Modern analytical workloads are highly heterogeneous and massively complex, making generic query optimizers untenable for many customers and scenarios. As a result, it is important to specialize these optimizers to instances of the workloads. In this paper, we continue a recent line of work in steering a query optimizer towards better plans for a given workload, and make major strides in pushing previous research ideas to production deployment. Along the way we solve several operational challenges including, making steering actions more manageable, keeping the costs of steering within budget, and avoiding unexpected performance regressions in production. Our resulting system, QQ-advisor, essentially externalizes the query planner to a massive offline pipeline for better exploration and specialization. We discuss various aspects of our design and show detailed results over production SCOPE workloads at Microsoft, where the system is currently enabled by default.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。