arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.03256cs.CL
收藏

从弱模型与强模型中合成文本到SQL数据

Synthesizing Text-to-SQL Data from Weak and Strong LLMs

Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Chinese Academy of Sciences(中国科学院大学);Alibaba Group(阿里巴巴集团)

Jiaxi Yang, Binyuan Hui, Min Yang, Jian Yang, Junyang Lin, Chang Zhou

AI总结:

本文提出结合强模型生成数据与弱模型错误信息的合成数据方法,用于指令微调开源LLMs,得到文本到SQL模型SENSE,在SPIDER和BIRD上达到最先进性能,缩小了与闭源模型方法的差距。

摘要:

开源与闭源大型语言模型(LLMs)之间的能力差距在文本到SQL任务中仍然是一个挑战。本文提出了一种合成数据方法,将更大、更强大的模型(强模型)生成的数据与较小、对齐不佳的模型(弱模型)生成的错误信息数据相结合。该方法不仅增强了文本到SQL模型的领域泛化能力,还通过偏好学习探索了错误数据监督的潜力。此外,我们将该合成数据方法用于开源LLMs的指令微调,得到了专门的文本到SQL模型SENSE。SENSE在SPIDER和BIRD基准上取得了最先进的结果,弥合了开源模型与由闭源模型提示的方法之间的性能差距。

英文摘要:

The capability gap between open-source and closed-source large language models (LLMs) remains a challenge in text-to-SQL tasks. In this paper, we introduce a synthetic data approach that combines data produced by larger, more powerful models (strong models) with error information data generated by smaller, not well-aligned models (weak models). The method not only enhances the domain generalization of text-to-SQL models but also explores the potential of error data supervision through preference learning. Furthermore, we employ the synthetic data approach for instruction tuning on open-source LLMs, resulting SENSE, a specialized text-to-SQL model. The effectiveness of SENSE is demonstrated through state-of-the-art results on the SPIDER and BIRD benchmarks, bridging the performance gap between open-source models and methods prompted by closed-source models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑