CAPER: 面向Text-to-SQL的子句对齐过程监督
CAPER: Clause-Aligned Process Supervision for Text-to-SQL
AI总结:
提出CAPER方法,通过反事实干预SQL抽象语法树自动推导子句级监督,训练轻量级Clause-PRM模型CAPER-9B,用于策略优化和候选验证,在BIRD和Spider数据集上提升了执行准确率和故障定位能力。
摘要:
Text-to-SQL系统通常通过查询级执行正确性进行评估,但这种终端信号对于哪个中间SQL决策导致成功或失败几乎没有指导作用。Token级密集监督也不适合:SQL token与完整的语义决策不对齐,可能惩罚执行等效的查询,并且难以大规模可靠标记。因此,我们提出CAPER,通过对SQL抽象语法树进行反事实干预自动推导子句级监督,实现用于奖励建模的根因错误定位;所得数据用于训练CAPER-9B,一个轻量级的Clause-PRM,为策略优化和候选验证提供子句边界反馈。在BIRD和Spider上的实验表明,子句对齐监督不仅提高了执行准确率(相对于GPT-5.4实现了高达15.3%的相对EX提升),还增强了故障定位能力,在保留的故障上达到了84.53%的准确率和90.60%的MRR。我们的项目页面位于此https URL。
英文摘要:
Text-to-SQL systems are typically evaluated by query-level execution correctness, but this terminal signal provides little guidance about which intermediate SQL decision caused success or failure. Token-level dense supervision is also ill-suited: SQL tokens do not align with complete semantic decisions, can penalize execution-equivalent queries, and are difficult to label reliably at scale. We therefore propose CAPER, which automatically derives clause-level supervision via counterfactual intervention on the SQL abstract syntax tree, enabling root-cause error localization for reward modeling; the resulting data is used to train CAPER-9B, a lightweight Clause-PRM that provides clause-boundary feedback for policy optimization and candidate verification. Experiments on BIRD and Spider show that clause-aligned supervision not only improves execution accuracy, achieving up to a 15.3% relative EX improvement over GPT-5.4, but also strengthens failure-localization capability, reaching 84.53% accuracy and 90.60% MRR on held-out failures. Our project page is at https://github.com/banrichard/RL-NL2SQL.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。