Text-to-SQL aims at generating SQL queries for the given natural language questions and thus helping users to query databases. Prompt learning with large language models (LLMs) has emerged as a recent approach, which designs prompts to lead LLMs to understand the input question and generate the corresponding SQL. However, it faces challenges with strict SQL syntax requirements. Existing work prompts the LLMs with a list of demonstration examples (i.e. question-SQL pairs) to generate SQL, but the fixed prompts can hardly handle the scenario where the semantic gap between the retrieved demonstration and the input question is large. In this paper, we propose a retrieval-augmented prompting method for a LLM-based Text-to-SQL framework, involving sample-aware prompting and a dynamic revision chain. Our approach incorporates sample-aware demonstrations, which include the composition of SQL operators and fine-grained information related to the given question. To retrieve questions sharing similar intents with input questions, we propose two strategies for assisting retrieval. Firstly, we leverage LLMs to simplify the original questions, unifying the syntax and thereby clarifying the users' intentions. To generate executable and accurate SQLs without human intervention, we design a dynamic revision chain which iteratively adapts fine-grained feedback from the previously generated SQL. Experimental results on three Text-to-SQL benchmarks demonstrate the superiority of our method over strong baseline models.
LLM-Based SQL Generation: Prompting, Self-Refinement, and Adaptive Weighted Majority Voting
Institute of Information Management(信息管理研究所);National Yang Ming Chiao Tung University(阳明交通大学);R. B. Annis School of Engineering(R. B. Annis工程学院);University of Indianapolis(印第安纳大学)
Structure Guided Large Language Model for SQL Generation
Department of Computing, The Hong Kong Polytechnic University, Hung Hom, Hong Kong SAR, China(香港理工大学计算机系);City University of Macau, Macau SAR, China(澳门城市大学);College of Information Science and Technology, Jinan University, GZ, China(广州jinan大学信息科学与技术学院)
Query Carefully: Detecting the Unanswerables in Text-to-SQL Tasks
Institute of Computer Science, Zurich University of Applied Sciences(计算机科学研究所,应用科学大学苏黎世);Institute of Medical Virology, University of Zurich(医学病毒学研究所,苏黎世大学);Department of Gastroenterology and Hepatology, University Hospital Zurich, University of Zurich(消化内科与肝病科,苏黎世大学医院,苏黎世大学)
Evaluating LLMs on Conversational Text-to-SQL under Chain Ambiguity and Intent Drift
The Hong Kong Polytechnic University(香港理工大学);City University of Macau(澳门城市大学);Jilin University(吉林大学);Beihang University(北京航空航天大学);Jinan University(暨南大学)
PET-SQL: A Prompt-Enhanced Two-Round Refinement of Text-to-SQL with Cross-consistency
SenseTime Research(商汤科技研究院);Peking University(北京大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);School of Computer Science and Technology, Beijing Institute of Technology(北京理工大学计算机学院);University of Cologne(科隆大学)
AskDB: An LLM Agent for Natural Language Interaction with Relational Databases
IT Operations, Mantu Group(Mantu集团信息技术部);Faculty of Engineering, Vietnamese-German University(越南-德国大学工程学院);Computer Science and Information Engineering, National Taiwan University(国立台湾大学计算机科学与工程系)
Empirical Characterization of Inference-Time Elicited Probability Transformations in Large Language Models
Bloch School of Management, Regnier Institute for Entrepreneurship & Innovation, University of Missouri–Kansas City(布洛赫管理学院、雷尼创业与创新研究所、密苏里大学堪萨斯城分校);Department of Computer Science, School of Science and Engineering, University of Missouri–Kansas City(计算机科学系、科学与工程学院、密苏里大学堪萨斯城分校)
Can Language Models Analyze Data? Evaluating Large Language Models for Question Answering over Datasets
School of Production Engineering and Management, Technical University of Crete(生产工程与管理学院,希腊克里特技术大学);Institute of Computer Science, Foundation for Research and Technology - Hellas(计算机科学研究所,希腊基础研究与技术研究院)
PentestEval: Benchmarking LLM-based Penetration Testing with Modular and Stage-Level Design
School of Computing and Information Systems, Singapore Management University(计算与信息系统学院,新加坡国立管理大学);School of Computer Science and Engineering, Nanyang Technological University(计算机科学与工程学院,南洋理工大学);School of Cyber Security, Tianjin University(网络安全学院,天津大学)