arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.20004cs.DBcs.CL
收藏

ReViSQL:实现人水平的文本到SQL

Human-Level Text-to-SQL via Reinforcement Learning on Verified Data, Without Pipeline Engineering

UIUC(伊利诺伊大学厄巴纳-香槟分校)

Yuxuan Zhu, Tengjun Jin, Yoojin Choi, Daniel Kang

AI总结:

本文提出ReViSQL框架,通过高质量训练数据提升SQL推理能力,首次在BIRD基准上达到人水平准确率,且在不同模型规模下均表现优异。

摘要:

将自然语言转换为SQL(文本到SQL)是数据库研究和数据分析应用中的关键挑战。最近的努力集中在通过开发大语言模型和人工智能代理来增强SQL推理能力,这些代理将文本到SQL任务分解为手动设计的、分步骤的流程。然而,尽管这些广泛的架构工程努力,仍然存在显著的差距:即使是最先进的(SOTA)人工智能代理仍未在BIRD基准上达到人水平的准确性。在本文中,我们表明,缩小这一差距不需要进一步的架构复杂性,而是需要干净的训练数据来提高底层模型的SQL推理能力。我们介绍了ReViSQL,这是一种流线化的框架,首次在BIRD基准上实现了人水平的准确性。与复杂的AI代理不同,ReViSQL利用了在BIRD-Verified上进行的强化学习与可验证奖励(RLVR),BIRD-Verified是一个我们编纂的数据集,其中包含基于BIRD训练集的2500个经过验证的文本到SQL实例。为了构建BIRD-Verified,我们设计了一个数据校正和验证工作流程,涉及SQL专家。我们识别并纠正了BIRD训练集子集中61.1%的数据错误。通过在BIRD-Verified上训练,我们证明了仅提高数据质量就能在相同的RLVR算法下将单次生成准确性提高8.2-13.9%。为了进一步提高性能,ReViSQL通过执行基于的 reconciliation 和多数投票进行推理时的缩放。经验上,我们通过两种模型规模展示了框架的优越性:ReViSQL-235B-A22B和ReViSQL-30B-A3B。在专家验证的BIRD Mini-Dev数据集上,ReViSQL-235B-A22B达到了93.2%的执行准确性,超过了代理人类水平准确性(92.96%),并且在先前的开源SOTA方法上提高了9.8%。我们的轻量级ReViSQL-30B-A3B在每查询成本上仅为之前的SOTA方法的7.5倍。

英文摘要:

Translating natural language questions to SQL queries (Text-to-SQL) is a long-standing problem in database research. Recent efforts have focused on improving accuracy by building increasingly complex multi-stage large LLM pipelines, layering task decomposition, schema linking, and model-based query selection on top of an LLM. Despite this growing complexity, a substantial gap (>10%) between such systems and human experts persists on benchmarks, suggesting that pipeline engineering alone has hit a ceiling. We show that human-level Text-to-SQL performance is achievable by fine-tuning an LLM using RLVR on clean data, without pipeline components. In this paper, we identified the dominant bottleneck for RLVR on Text-to-SQL: existing training data contains pervasive annotation errors that mislead optimization. To address this, we developed a multi-round, expert-driven verification pipeline and used it to curate BIRD-Platinum, a dataset of 2.5k verified instances sampled from BIRD Train, with errors corrected in 61% of instances. We show that fine-tuning Qwen3-235B on BIRD-Platinum yields consistent improvements (11-16%) over BIRD Train on Arcwise-Plat (an expert-verified version of BIRD) and Spider2, outperforming SOTA open-source systems by 0.6-16%. Furthermore, we diagnosed two failure modes of standard RLVR on Text-to-SQL. We find that (1) result-based rewards have non-trivial false positive rates, and (2) models systematically ignore the external knowledge in BIRD-style problems. To address them, we propose ReViSQL-BIRD, a specialized reward shaping method that combines result-based rewards with SQL equivalence verification and leverages process rewards for incentivizing external-knowledge use. We fine-tuned Kimi-K2.6 with ReViSQL-BIRD. On Arcwise-Plat, ReViSQL-BIRD-K2.6 is the first method to achieve human-level accuracy (92.96%), outperforming top five open-source systems by 10-22%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑