arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2009.13845cs.CLcs.AI
收藏

GraPPa:表语义解析的语法增强预训练

GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing

Salesforce Research(赛富时研究院);Yale University(耶鲁大学);University of Edinburgh(爱丁堡大学);Facebook AI(脸书人工智能研究院)

Tao Yu, Chien-Sheng Wu, Xi Victoria Lin, Bailin Wang, Yi Chern Tan, Xinyi Yang, Dragomir Radev, Richard Socher, Caiming Xiong

摘要:

我们提出了一种有效的预训练方法,用于表语义解析,学习文本和表格数据的联合表示中的组合归纳偏差。我们通过一个同步的上下文无关文法(SCFG)从现有的文本到SQL数据集诱导高质量的表上构建合成的问题SQL对。我们使用一种新的文本模式链接目标对合成数据预训练我们的模型,该目标预测每个问题-SQL对的SQL中表字段的语法角色。为了保持模型表示真实世界数据的能力,我们还在几个现有的表和语言数据集上包含了掩码语言建模(MLM),以规范预训练过程。在四个流行的全监督和弱监督表语义解析基准测试中,GraPPa作为特征表示层的性能明显优于Roberta-large,并在所有这些测试中建立了新的最先进的结果。

英文摘要:

We present GraPPa, an effective pre-training approach for table semantic parsing that learns a compositional inductive bias in the joint representations of textual and tabular data. We construct synthetic question-SQL pairs over high-quality tables via a synchronous context-free grammar (SCFG) induced from existing text-to-SQL datasets. We pre-train our model on the synthetic data using a novel text-schema linking objective that predicts the syntactic role of a table field in the SQL for each question-SQL pair. To maintain the model's ability to represent real-world data, we also include masked language modeling (MLM) over several existing table-and-language datasets to regularize the pre-training process. On four popular fully supervised and weakly supervised table semantic parsing benchmarks, GraPPa significantly outperforms RoBERTa-large as the feature representation layers and establishes new state-of-the-art results on all of them.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑