arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1908.11052cs.CLcs.LG
收藏

带辅助任务的文本到SQL零Zero-shot学习

Zero-shot Text-to-SQL Learning with Auxiliary Task

The Ohio State University(俄亥俄州立大学);Fudan University(复旦大学);JD.COM AI Research(京东AI研究院)

Shuaichen Chang, Pengfei Liu, Yun Tang, Jing Huang, Xiaodong He, Bowen Zhou

摘要:

近年来,在文本到SQL任务中使用神经seq2seq模型取得了巨大的成功。然而,很少有工作关注这些模型如何推广到现实中看不见的数据,这自然提出了一个问题:这种令人印象深刻的性能是否意味着一个完美的泛化模型,或者仍然存在一些限制?在本文中,我们首先通过提供一个新的测试平台来诊断文本到SQL任务的瓶颈,在该测试平台中,我们观察到现有模型对罕见数据的泛化能力很差。上述分析鼓励我们设计一个简单但有效的辅助任务,作为生成任务的支持模型和正则化项,以提高模型的泛化能力。实验上,我们评估我们的模型上的一个大型的文本到SQL数据集WikiSQL。与强大的基线粗到细模型相比,我们的模型在整个数据集上的绝对准确度提高了3%以上。更有趣的是,在WikiSQL的zero-shot子集测试中,我们的模型在基线上实现了5%的绝对准确率增益,清楚地证明了其优越的泛化能力。

英文摘要:

Recent years have seen great success in the use of neural seq2seq models on the text-to-SQL task. However, little work has paid attention to how these models generalize to realistic unseen data, which naturally raises a question: does this impressive performance signify a perfect generalization model, or are there still some limitations? In this paper, we first diagnose the bottleneck of text-to-SQL task by providing a new testbed, in which we observe that existing models present poor generalization ability on rarely-seen data. The above analysis encourages us to design a simple but effective auxiliary task, which serves as a supportive model as well as a regularization term to the generation task to increase the models generalization. Experimentally, We evaluate our models on a large text-to-SQL dataset WikiSQL. Compared to a strong baseline coarse-to-fine model, our models improve over the baseline by more than 3% absolute in accuracy on the whole dataset. More interestingly, on a zero-shot subset test of WikiSQL, our models achieve 5% absolute accuracy gain over the baseline, clearly demonstrating its superior generalizability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑