arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1902.01069cs.CL
收藏

基于表意识词上下文化的WikiSQL综合研究

A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization

Clova AI(Clova人工智能);NAVER Corp(NAVER公司)

Wonseok Hwang, Jinyeong Yim, Seunghyun Park, Minjoon Seo

摘要:

我们提出了SQLova,第一个自然语言到SQL(NL 2SQL)模型,以实现人类在WikiSQL数据集的性能。我们重新审视并讨论NL 2SQL文献中的各种流行方法,充分利用BERT {Devlin et al.,2018),并将它们连贯地结合起来,在逻辑形式和执行准确性上分别优于之前的最新技术水平8.2%和2.5%。我们特别注意到,带有seq 2seq解码器的BERT会导致任务性能不佳,这表明在使用这种大型预训练模型时仔细设计的重要性。我们还对数据集和模型进行了全面的分析,这有助于设计未来的NL 2SQL数据集和模型。我们特别指出,我们的模型的性能接近WikiSQL的上限,我们观察到很大一部分评估错误是由于错误的注释,我们的模型在执行准确性方面已经超过人类性能1.3%。

英文摘要:

We present SQLova, the first Natural-language-to-SQL (NL2SQL) model to achieve human performance in WikiSQL dataset. We revisit and discuss diverse popular methods in NL2SQL literature, take a full advantage of BERT {Devlin et al., 2018) through an effective table contextualization method, and coherently combine them, outperforming the previous state of the art by 8.2% and 2.5% in logical form and execution accuracy, respectively. We particularly note that BERT with a seq2seq decoder leads to a poor performance in the task, indicating the importance of a careful design when using such large pretrained models. We also provide a comprehensive analysis on the dataset and our model, which can be helpful for designing future NL2SQL datsets and models. We especially show that our model's performance is near the upper bound in WikiSQL, where we observe that a large portion of the evaluation errors are due to wrong annotations, and our model is already exceeding human performance by 1.3% in execution accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑