Exploring the generalization of a text-to-SQL parser is essential for a system to automatically adapt the real-world databases. Previous works provided investigations focusing on lexical diversity, including the influence of the synonym and perturbations in both natural language questions and databases. However, research on the structure variety of database schema~(DS) is deficient. Specifically, confronted with the same input question, the target SQL is probably represented in different ways when the DS comes to a different structure. In this work, we provide in-deep discussions about the structural generalization of text-to-SQL tasks. We observe that current datasets are too templated to study structural generalization. To collect eligible test data, we propose a framework to generate novel text-to-SQL data via automatic and synchronous (DS, SQL) pair altering. In the experiments, significant performance reduction when evaluating well-trained text-to-SQL models on the synthetic samples demonstrates the limitation of current research regarding structural generalization. According to comprehensive analysis, we suggest the practical reason is the overfitting of (NL, SQL) patterns.
TRUST-SQL: Tool-Integrated Multi-Turn Reinforcement Learning for Text-to-SQL over Unknown Schemas
Beijing University of Posts and Telecommunications(北京邮电大学);State Key Lab of Processors, Institute of Computing Technology, CAS(中国科学院计算技术研究所处理器国家重点实验室);University of Chinese Academy of Sciences(中国科学院大学);Meituan(美团)
SQL-o1: Automatic Text-to-SQL Program Synthesis via Self-Reward Heuristic Search
School of Computer Science, Beijing University of Posts and Telecommunications, China(北京邮电大学计算机科学学院);College of Computing and Data Science, Nanyang Technological University, Singapore(南洋理工大学计算机与数据科学学院);State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, China(北京邮电大学网络与交换技术国家重点实验室);College of Computer Science, Sichuan University, Chengdu, China(四川大学计算机学院)
A Survey of Large Language Model-Based Generative AI for Text-to-SQL: Benchmarks, Applications, Use Cases, and Challenges
Cleveland State University(克利夫兰州立大学);The Davey Tree Expert Company(戴维树专家公司);The MathWorks(MathWorks公司);Khoury College of Computer Science, Roux Institute at Northeastern University(东北大学鲁克斯研究所科里计算机科学学院)
LLM-Based SQL Generation: Prompting, Self-Refinement, and Adaptive Weighted Majority Voting
Institute of Information Management(信息管理研究所);National Yang Ming Chiao Tung University(阳明交通大学);R. B. Annis School of Engineering(R. B. Annis工程学院);University of Indianapolis(印第安纳大学)
Query Carefully: Detecting the Unanswerables in Text-to-SQL Tasks
Institute of Computer Science, Zurich University of Applied Sciences(计算机科学研究所,应用科学大学苏黎世);Institute of Medical Virology, University of Zurich(医学病毒学研究所,苏黎世大学);Department of Gastroenterology and Hepatology, University Hospital Zurich, University of Zurich(消化内科与肝病科,苏黎世大学医院,苏黎世大学)
Beyond Static Rules: Automated Discovery of Latent Vulnerabilities in Text-to-SQL
Shanghai University of Finance and Economics(上海财经大学);Beihang University(北京航空航天大学);Deepexi Technology Co. Ltd.(深圳市智元机器有限公司);Southern University of Science and Technology(南方科技大学)
Database Context Compression for Text-to-SQL on Real-World Large Databases
National Engineering Research Center of Software Engineering, Peking University(软件工程国家工程研究中心,北京大学);School of Computer Science, Peking University(北京大学计算机学院);Peking University Information Technology Institute, Tianjin Binhai(天津滨海大学信息技术研究所,北京大学)
Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);Beijing Key Laboratory of Research on Large Models and Intelligent Governance(北京大型模型与智能治理研究重点实验室);Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE(下一代智能搜索与推荐工程研究中心,教育部);Tianjin University of Science and Technology(天津科技大学);Beijing Institute of Technology(北京理工大学)
Bilinear Classes: A Structural Framework for Provable Generalization in RL
University of Washington(华盛顿大学);Microsoft Research(微软研究院);Princeton University(普林斯顿大学);University of California, San Diego(加利福尼亚大学圣迭戈分校);Cornell University(康奈尔大学);Carnegie Mellon University(卡内基梅隆大学)