arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.09060cs.DB
收藏

GenRewrite: 通过大语言模型实现查询重写

GenRewrite: Query Rewriting via Large Language Models

University of Michigan(密歇根大学)

Jie Liu, Barzan Mozafari

AI总结:

GenRewrite 利用大语言模型实现查询重写,通过自然语言重写规则和反例引导技术提升查询优化效率。

摘要:

查询重写是一种在查询到达查询优化器之前优化 poorly written queries 的有效技术。然而,手动重写不可扩展,因为它容易出错且需要深厚的专 expertise。传统的查询重写算法也存在不足:基于规则的方法无法泛化到新的查询模式,而合成方法在处理复杂查询时遇到困难。幸运的是,大语言模型(LLMs)已经具备广泛的知识和先进的推理能力,使它们成为解决这些长期挑战的有希望的解决方案。在本文中,我们提出了 GenRewrite,这是首个利用 LLMs 进行查询重写超越传统规则的综合系统。我们引入了自然语言重写规则(NLR2s)的概念,这些规则不仅为 LLM 提供提示,还是一种从重写一个查询到另一个查询的知识转移手段,使 GenRewrite 能够随着时间变得更聪明和更有效。我们提出了一种新颖的反例引导技术,该技术通过迭代修正重写查询中的语法和语义错误,显著降低了 LLM 的成本和手动验证所需的努力。在标准 TPC-DS 和 JOB 基准及其 SQLStorm 生成的变体上,GenRewrite 在每个加速阈值下都比所有基线优化了更多的查询。在 TPC-DS 上的 >=2x 阈值,GenRewrite 比 LLM 驱动的基线改进了 25 个查询-1.35 倍,比 LLM 增强的基于规则的基线改进了 2.6 倍,并且在 TPC-DS(SQLStorm)上差距进一步扩大;在 JOB 和其 SQLStorm 变体上,查询更简单,绝对收益较小,但 GenRewrite 仍然以显著优势领先。

英文摘要:

Query rewriting is an effective technique for refining poorly written queries before they reach the query optimizer. However, manual rewriting is not scalable, as it is prone to errors and requires deep expertise. Traditional query rewriting algorithms fall short too: rule-based approaches fail to generalize to new query patterns, while synthesis-based methods struggle with complex queries. Fortunately, Large Language Models (LLMs) already possess broad knowledge and advanced reasoning capabilities, making them a promising solution for tackling these longstanding challenges. In this paper, we present GenRewrite, the first holistic system that leverages LLMs for query rewriting beyond traditional rules. We introduce the notion of Natural Language Rewrite Rules (NLR2s), which serve as hints for the LLM while also a means of knowledge transfer from rewriting one query to another, allowing GenRewrite to become smarter and more effective over time. We present a novel counterexample-guided technique that iteratively corrects the syntactic and semantic errors in the rewritten query, significantly reducing the LLM costs and the manual effort required for verification. Across the standard TPC-DS and JOB benchmarks and their SQLStorm-generated variants, GenRewrite consistently optimizes more queries at every speedup threshold than all baselines. At the >=2x threshold on TPC-DS, GenRewrite improves 25 queries-1.35x more than LLM-driven baselines and 2.6x more than LLM-enhanced rule-based baselines-and the gap widens further on TPC-DS (SQLStorm); on JOB and its SQLStorm variant, where queries are simpler, absolute gains are smaller but GenRewrite still leads by a notable margin.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑