arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.07930cs.CLcs.AI
收藏

MAG-SQL:基于软模式链接与迭代子SQL细化的多智能体生成式文本到SQL方法

MAG-SQL: Multi-Agent Generative Approach with Soft Schema Linking and Iterative Sub-SQL Refinement for Text-to-SQL

South China University of Technology(华南理工大学);Tsinghua University(清华大学)

Wenxuan Xie, Gaochen Wu, Bowen Zhou

AI总结:

针对复杂数据库模式与问题下文本到SQL性能不足及中间步骤缺乏监督的问题,提出MAG-SQL多智能体生成方法,采用软模式链接与迭代子SQL细化,在BIRD上以GPT-4实现61.08%执行准确率,超越基线。

摘要:

近期基于上下文学习的方法在文本到SQL任务中取得了显著成功。然而,在具有复杂数据库模式和困难问题的数据集(如BIRD)上,这些模型的性能与人类表现之间仍存在较大差距。此外,现有工作在采用问题分解方法迭代求解时忽视了对中间步骤的监督,且这些工作中使用的模式链接方法非常粗糙。为解决这些问题,我们提出了MAG-SQL,一种结合软模式链接和迭代子SQL细化的多智能体生成式方法。在我们的框架中,采用基于实体并利用表摘要的方法来选择数据库中的列,并引入一种新颖的目标-条件分解方法来分解那些复杂问题。此外,我们构建了一个迭代生成模块,包含子SQL生成器和子SQL细化器,为生成的每一步引入外部监督。通过一系列消融研究,我们证明了框架中每个智能体的有效性。在BIRD基准上使用GPT-4进行评估时,MAG-SQL达到了61.08%的执行准确率,而原始GPT-4的基线准确率为46.35%,MAC-SQL的基线准确率为57.56%。此外,我们的方法在Spider上也取得了类似的进展。代码可在https://github.com/LancelotXWX/MAG-SQL获取。

英文摘要:

Recent In-Context Learning based methods have achieved remarkable success in Text-to-SQL task. However, there is still a large gap between the performance of these models and human performance on datasets with complex database schema and difficult questions, such as BIRD. Besides, existing work has neglected to supervise intermediate steps when solving questions iteratively with question decomposition methods, and the schema linking methods used in these works are very rudimentary. To address these issues, we propose MAG-SQL, a multi-agent generative approach with soft schema linking and iterative Sub-SQL refinement. In our framework, an entity-based method with tables' summary is used to select the columns in database, and a novel targets-conditions decomposition method is introduced to decompose those complex questions. Additionally, we build a iterative generating module which includes a Sub-SQL Generator and Sub-SQL Refiner, introducing external oversight for each step of generation. Through a series of ablation studies, the effectiveness of each agent in our framework has been demonstrated. When evaluated on the BIRD benchmark with GPT-4, MAG-SQL achieves an execution accuracy of 61.08%, compared to the baseline accuracy of 46.35% for vanilla GPT-4 and the baseline accuracy of 57.56% for MAC-SQL. Besides, our approach makes similar progress on Spider. The codes are available at https://github.com/LancelotXWX/MAG-SQL.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑