arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.04282cs.DBcs.PL
收藏

自适应递归查询优化

Adaptive Recursive Query Optimization

Anna Herlihy, Guillaume Martres, Anastasia Ailamaki, Martin Odersky

AI总结:

本文提出自适应元编程技术,将递归查询优化移至运行时,通过动态重优化和自定义连接排序,在Carac引擎上实现未优化查询提速千倍、手工优化查询提速6倍。

摘要:

性能关键的工业应用,包括大规模程序分析、网络分析和分布式系统分析,越来越依赖于递归查询进行数据分析。然而,传统的基于关系代数的查询优化技术难以很好地扩展到递归查询处理,这是因为查询评估的迭代性质,在单次查询执行过程中,关系基数可能不可预测地变化。为了避免容易出错的基数估计,自适应查询处理技术利用运行时信息来指导查询优化,但这些系统并未针对递归查询处理的特定需求进行优化。在本文中,我们引入了自适应元编程(Adaptive Metaprogramming),这是一种创新技术,通过原则性的元编程将递归查询优化和代码生成从编译时转移到运行时,从而在查询执行开始之前和之后实现动态优化和重新优化。我们提出了一种自定义的连接排序优化,适用于查询编译和执行过程中的多个阶段。通过自定义的Datalog引擎Carac,我们评估了自适应元编程的优化潜力,并展示了未优化的递归查询执行时间可以提高三个数量级,而手工优化的查询可以提高6倍。

英文摘要:

Performance-critical industrial applications, including large-scale program, network, and distributed system analyses, are increasingly reliant on recursive queries for data analysis. Yet traditional relational algebra-based query optimization techniques do not scale well to recursive query processing due to the iterative nature of query evaluation, where relation cardinalities can change unpredictably during the course of a single query execution. To avoid error-prone cardinality estimation, adaptive query processing techniques use runtime information to inform query optimization, but these systems are not optimized for the specific needs of recursive query processing. In this paper, we introduce Adaptive Metaprogramming, an innovative technique that shifts recursive query optimization and code generation from compile-time to runtime using principled metaprogramming, enabling dynamic optimization and re-optimization before and after query execution has begun. We present a custom join-ordering optimization applicable at multiple stages during query compilation and execution. Through Carac, a custom Datalog engine, we evaluate the optimization potential of Adaptive Metaprogramming and show unoptimized recursive query execution time can be improved by three orders of magnitude and hand-optimized queries by 6x.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑