大型语言模型之战：Dolly vs LLaMA vs Vicuna vs Guanaco vs Bard vs ChatGPT——文本到SQL解析比较
Battle of the Large Language Models: Dolly vs LLaMA vs Vicuna vs Guanaco vs Bard vs ChatGPT -- A Text-to-SQL Parsing Comparison
AI总结：
本研究系统比较六种大型语言模型在文本到SQL解析任务上的表现，发现开源模型性能显著落后于闭源模型，需进一步工作弥合差距。
摘要：
ChatGPT的成功引发了一场人工智能竞赛，研究人员致力于开发能够匹敌或超越商业模型语言理解与生成能力的新型大型语言模型（LLM）。近期，多个模型通过不同的指令微调方法声称性能接近GPT-3.5或GPT-4。作为文本到SQL解析的实践者，我们感谢他们对开源研究的宝贵贡献。然而，以审慎的态度审视这些声明并确证这些模型的实际效果至关重要。因此，我们让六个流行的大型语言模型相互较量，在九个基准数据集上使用五种不同的提示策略，系统评估其文本到SQL解析能力，覆盖零样本和少样本场景。遗憾的是，开源模型的性能显著落后于GPT-3.5等闭源模型，凸显了进一步工作以弥合这些模型间性能差距的必要性。
英文摘要：
The success of ChatGPT has ignited an AI race, with researchers striving to develop new large language models (LLMs) that can match or surpass the language understanding and generation abilities of commercial ones. In recent times, a number of models have emerged, claiming performance near that of GPT-3.5 or GPT-4 through various instruction-tuning methods. As practitioners of Text-to-SQL parsing, we are grateful for their valuable contributions to open-source research. However, it is important to approach these claims with a sense of scrutiny and ascertain the actual effectiveness of these models. Therefore, we pit six popular large language models against each other, systematically evaluating their Text-to-SQL parsing capability on nine benchmark datasets with five different prompting strategies, covering both zero-shot and few-shot scenarios. Regrettably, the open-sourced models fell significantly short of the performance achieved by closed-source models like GPT-3.5, highlighting the need for further work to bridge the performance gap between these models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
