Since we can leverage a large amount of unlabeled data without any human supervision to train a model and transfer the knowledge to target tasks, self-supervised learning is a de-facto component for the recent success of deep learning in various fields. However, in many cases, there is a discrepancy between a self-supervised learning objective and a task-specific objective. In order to tackle such discrepancy in Text-to-SQL task, we propose a novel self-supervised learning framework. We utilize the task-specific properties of Text-to-SQL task and the underlying structures of table contents to train the models to learn useful knowledge of the \textit{header-column} alignment task from unlabeled table data. We are able to transfer the knowledge to the supervised Text-to-SQL training with annotated samples, so that the model can leverage the knowledge to better perform the \textit{header-span} alignment task to predict SQL statements. Experimental results show that our self-supervised learning framework significantly improves the performance of the existing strong BERT based models without using large external corpora. In particular, our method is effective for training the model with scarce labeled data. The source code of this work is available in GitHub.
Measuring and Improving Compositional Generalization in Text-to-SQL via Component Alignment
Queen Mary University of London（伦敦玛丽女王大学）；UC Berkeley（加州大学伯克利分校）；Jožef Stefan Institute（约瑟夫·斯特凡研究所）；Nanning Central Sub-branch of the People’s Bank of China（中国人民银行南宁中心支行）
The Ohio State University（俄亥俄州立大学）；Microsoft Research（微软研究院）
学习捕获文本表对齐对于像文本到SQL这样的任务是必不可少的。模型需要正确识别对列和值的自然语言引用，并将它们置于给定的数据库模式中。在本文中，我们提出了一种新的弱监督结构接地预训练框架（StruG）的文本到SQL，可以有效地学习捕获基于并行文本表语料库的文本表对齐。我们确定了一组新的预测任务：列接地，值接地和列值映射，并利用它们来预训练文本表编码器。此外，为了在更现实的文本表对齐设置下评估不同的方法，我们基于Spider dev set创建了一个新的评估集Spider-Realistic，其中删除了明确提到的列名，并采用八个现有的文本到SQL数据集进行跨数据库评估。在所有设置中，BERT-LARGE带来了显着的改进。与现有的预训练方法（如GRAPPA）相比，BLOG在Spider上实现了类似的性能，并且在更真实的集合上优于所有基线。Spider-Realistic数据集可在https://doi.org/10.5281/zenodo.5205322上获得。
$R^3$-NL2GQL: A Model Coordination and Knowledge Graph Alignment Approach for NL2GQL
School of Computer Science, Fudan University（复旦大学计算机科学学院）；Institute of Fintech, Fudan University（复旦大学金融科技研究院）；School of Electronics and Information Engineering, Tongji University（同济大学电子与信息工程学院）；Tandon School of Engineering, New York University（纽约大学坦登工程学院）；DataGrand Inc.（数据堂公司）
