arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.10671cs.CL
收藏

通过分解增强注意力:利用工作流范式改进基于大语言模型的文本到SQL

Decomposition for Enhancing Attention: Improving LLM-based Text-to-SQL through Workflow Paradigm

Sun Yat-sen University(中山大学);Platform and Content Group, Tencent(腾讯平台与内容事业群)

Yuanzhen Xie, Xinzhou Jin, Tao Xie, MingXiong Lin, Liang Chen, Chenyun Yu, Lei Cheng, ChengXiang Zhuo, Bo Hu, Zang Li

AI总结:

本文提出一种基于工作流范式的文本到SQL方法,通过信息确定、问题分类、自校正和主动学习等模块增强大语言模型的注意力与求解范围,在多个数据集上取得显著提升。

摘要:

大语言模型(LLMs)的上下文学习在自然语言处理领域取得了显著成功,而大量案例研究表明,单步链式思维提示方法面临注意力扩散以及在文本到SQL等复杂任务中性能不足等挑战。为了提升大语言模型在文本到SQL中的上下文学习能力,本文提出了一种工作流范式方法,旨在通过分解来增强大语言模型的注意力并扩大其问题求解范围。具体而言,用于消除冗余信息的信息确定模块以及基于问题分类的全新提示结构极大地增强了模型的注意力。此外,引入自校正和主动学习模块极大地扩展了大语言模型的问题求解范围,从而提高了基于大语言模型方法的上限。在三个数据集上进行的大量实验表明,我们的方法显著优于其他方法。在Spider Dev、Spider-Realistic和Bird Dev数据集上,与现有基线相比取得了约2-3个百分点的提升,并在Spider Test数据集上取得了新的SOTA结果。我们的代码已在GitHub上公开:\url{https://github.com/FlyingFeather/DEA-SQL}。

英文摘要:

In-context learning of large-language models (LLMs) has achieved remarkable success in the field of natural language processing, while extensive case studies reveal that the single-step chain-of-thought prompting approach faces challenges such as attention diffusion and inadequate performance in complex tasks like text-to-SQL. To improve the contextual learning capabilities of LLMs in text-to-SQL, a workflow paradigm method is proposed, aiming to enhance the attention and problem-solving scope of LLMs through decomposition. Specifically, the information determination module for eliminating redundant information and the brand-new prompt structure based on problem classification greatly enhance the model's attention. Additionally, the inclusion of self-correction and active learning modules greatly expands the problem-solving scope of LLMs, hence improving the upper limit of LLM-based approaches. Extensive experiments conducted on three datasets demonstrate that our approach outperforms other methods by a significant margin. About 2-3 percentage point improvements compared to the existing baseline on the Spider Dev, Spider-Realistic, and Bird Dev datasets and new SOTA results on the Spider Test dataset are achieved. Our code is available on GitHub: \url{https://github.com/FlyingFeather/DEA-SQL}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑