Natural language interfaces to databases (NLIDB) democratize end user access to relational data. Due to fundamental differences between natural language communication and programming, it is common for end users to issue questions that are ambiguous to the system or fall outside the semantic scope of its underlying query language. We present Photon, a robust, modular, cross-domain NLIDB that can flag natural language input to which a SQL mapping cannot be immediately determined. Photon consists of a strong neural semantic parser (63.2\% structure accuracy on the Spider dev benchmark), a human-in-the-loop question corrector, a SQL executor and a response generator. The question corrector is a discriminative neural sequence editor which detects confusion span(s) in the input question and suggests rephrasing until a translatable input is given by the user or a maximum number of iterations are conducted. Experiments on simulated data show that the proposed method effectively improves the robustness of text-to-SQL system against untranslatable user input. The live demo of our system is available at http://naturalsql.com.
Recently, context-dependent text-to-SQL semantic parsing which translates natural language into SQL in an interaction process has attracted a lot of attention. Previous works leverage context-dependence information either from interaction history utterances or the previous predicted SQL queries but fail in taking advantage of both since of the mismatch between natural language and logic-form SQL. In this work, we propose a History Information Enhanced text-to-SQL model (HIE-SQL) to exploit context-dependence information from both history utterances and the last predicted SQL query. In view of …
$R^3$: "This is My SQL, Are You With Me?" A Consensus-Based Multi-Agent System for Text-to-SQL Tasks
School of Engineering, Westlake University（西湖大学工程学院）；University of Michigan（密歇根大学）；Carnegie Mellon University（卡内基梅隆大学）；School of Mathematical Science, Shanghai Jiao Tong University（上海交通大学数学科学学院）
Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL
Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算机系）；Faculty of Data Science, City University of Macau（澳门城市大学数据科学学院）；College of Cyber Security, Jinan University（暨南大学网络安全学院）
Harbin Institute of Technology（哈尔滨工业大学）；Microsoft Research Asia（微软亚洲研究院）
Existing text-to-SQL semantic parsers are typically designed for particular settings such as handling queries that span multiple tables, domains or turns which makes them ineffective when applied to different settings. We present UniSAr (Unified Structure-Aware Autoregressive Language Model), which benefits from directly using an off-the-shelf language model architecture and demonstrates consistently high performance under different settings. Specifically, UniSAr extends existing autoregressive language models to incorporate three non-invasive extensions to make them structure-aware: (1) addin…
The University of Michigan, Ann Arbor（密歇根大学安娜堡分校）；The Ohio State University（俄亥俄州立大学）；The University of Waterloo（滑铁卢大学）；The University of Hong Kong（香港大学）；Penn State University（宾夕法尼亚州立大学）
Existing text-to-SQL research only considers complete questions as the input, but lay-users might strive to formulate a complete question. To build a smarter natural language interface to database systems (NLIDB) that also processes incomplete questions, we propose a new task, prefix-to-SQL which takes question prefix from users as the input and predicts the intended SQL. We construct a new benchmark called PAGSAS that contains 124K user question prefixes and the intended SQL for 5 sub-tasks Advising, GeoQuery, Scholar, ATIS, and Spider. Additionally, we propose a new metric SAVE to measure ho…
Pi-SQL: Enhancing Text-to-SQL with Fine-Grained Guidance from Pivot Programming Languages
Shanghai University of Finance and Economics（上海金融学院）；Tsinghua University（清华大学）；Beihang University（北航）；Wuhan University（武汉大学）；Southern University of Science and Technology（南方科技大学）
Conversational text-to-SQL is designed to translate multi-turn natural language questions into their corresponding SQL queries. Most state-of-the-art conversational text- to-SQL methods are incompatible with generative pre-trained language models (PLMs), such as T5. In this paper, we present a two-stage unified MultI-task Generation frAmework (MIGA) that leverages PLMs' ability to tackle conversational text-to-SQL. In the pre-training stage, MIGA first decomposes the main task into several related sub-tasks and then unifies them into the same sequence-to-sequence (Seq2Seq) paradigm with task-s…
SAP Labs Korea（SAP韩国实验室）；Seoul National University（首尔国立大学）
Most deep learning approaches for text-to-SQL generation are limited to the WikiSQL dataset, which only supports very simple queries. Recently, template-based and sequence-to-sequence approaches were proposed to support complex queries, which contain join queries, nested queries, and other types. However, Finegan-Dollak et al. (2018) demonstrated that both the approaches lack the ability to generate SQL of unseen templates. In this paper, we propose a template-based one-shot learning model for the text-to-SQL generation so that the model can generate SQL of an untrained template based on a sin…
From Natural Language to SQL: Review of LLM-based Text-to-SQL Systems
School of Computing and Informatics, University of Louisiana at Lafayette（路易斯安那大学拉斐特分校计算与信息学院）；School of Mechanical Engineering, University of Louisiana at Lafayette（路易斯安那大学拉斐特分校机械工程学院）
Conversational text-to-SQL aims at converting multi-turn natural language queries into their corresponding SQL (Structured Query Language) representations. One of the most intractable problems of conversational text-to-SQL is modelling the semantics of multi-turn queries and gathering the proper information required for the current query. This paper shows that explicitly modelling the semantic changes by adding each turn and the summarization of the whole context can bring better performance on converting conversational queries into SQLs. In particular, we propose two conversational modelling …
The task of text-to-SQL parsing, which aims at converting natural language questions into executable SQL queries, has garnered increasing attention in recent years, as it can assist end users in efficiently extracting vital information from databases without the need for technical background. One of the major challenges in text-to-SQL parsing is domain generalization, i.e., how to generalize well to unseen databases. Recently, the pre-trained text-to-text transformer model, namely T5, though not specialized for text-to-SQL parsing, has achieved state-of-the-art performance on standard benchmar…
SQLBench: A Comprehensive Evaluation for Text-to-SQL Capabilities of Large Language Models
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；SenseTime Research（商汤科技研究院）；School of Artificial Intelligence, Shandong University（山东大学人工智能学院）；School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；Technical University of Munich, Heilbronn Data Science Center, Munich Data Science Institute（慕尼黑技术大学，海德堡数据科学中心，慕尼黑数据科学研究所）；Institute of Microelectronics, Chinese Academy of Sciences（中国科学院微电子研究所）
ErrorLLM: Modeling SQL Errors for Text-to-SQL Refinement
The Hong Kong Polytechnic University（香港理工大学）；City University of Macau（澳门城市大学）；Jilin University（吉林大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Beihang University（北航大学）
The translation of natural language questions to SQL queries has attracted growing attention, in particular in connection with transformers and similar language models. A large number of techniques are geared towards the English language; in this work, we thus investigated translation to SQL when input questions are given in the Portuguese language. To do so, we properly adapted state-of-the-art tools and resources. We changed the RAT-SQL+GAP system by relying on a multilingual BART model (we report tests with other language models), and we produced a translated version of the Spider dataset. …
University of Edinburgh（爱丁堡大学）；UC Berkeley（加州大学伯克利分校）；Microsoft Research（微软研究院）
When translating natural language questions into SQL queries to answer questions from a database, contemporary semantic parsing models struggle to generalize to unseen database schemas. The generalization challenge lies in (a) encoding the database relations in an accessible way for the semantic parser, and (b) modeling alignment between database columns and their mentions in a given query. We present a unified framework, based on the relation-aware self-attention mechanism, to address schema encoding, schema linking, and feature representation within a text-to-SQL encoder. On the challenging …
Most existing studies in text-to-SQL tasks do not require generating complex SQL queries with multiple clauses or sub-queries, and generalizing to new, unseen databases. In this paper we propose SyntaxSQLNet, a syntax tree network to address the complex and cross-domain text-to-SQL generation task. SyntaxSQLNet employs a SQL specific syntax tree-based decoder with SQL generation path history and table-aware column attention encoders. We evaluate SyntaxSQLNet on the Spider text-to-SQL task, which contains databases with multiple tables and complex SQL queries with multiple SQL clauses and neste…
Tsinghua University（清华大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）
This paper presents the first comprehensive analysis of ChatGPT's Text-to-SQL ability. Given the recent emergence of large-scale conversational language model ChatGPT and its impressive capabilities in both conversational abilities and code generation, we sought to evaluate its Text-to-SQL performance. We conducted experiments on 12 benchmark datasets with different languages, settings, or scenarios, and the results demonstrate that ChatGPT has strong text-to-SQL abilities. Although there is still a gap from the current state-of-the-art (SOTA) model performance, considering that the experiment…
