基于结构化思维链的知识蒸馏用于文本到SQL
Knowledge Distillation with Structured Chain-of-Thought for Text-to-SQL
AI总结：
本文提出Struct-SQL框架，通过结构化思维链知识蒸馏提升小型语言模型在文本到SQL任务中的性能，实现8.1%的准确率提升。
摘要：
在企业层面部署准确的文本到SQL系统面临一个困难的三重困境，涉及成本、安全性和性能。当前解决方案迫使企业在昂贵的专有大型语言模型（LLMs）和低性能的小型语言模型（SLMs）之间做出选择。努力改进SLMs通常依赖于通过无结构的思维链（CoT）轨迹从大型LLMs中蒸馏推理，这一过程仍然本质上是模糊的。相反，我们假设一个正式的、结构化的推理表示能提供更清晰、更可靠的教学信号，因为文本到SQL任务需要明确且精确的逻辑步骤。为了验证这一假设，我们提出了Struct-SQL，一种新的知识蒸馏（KD）框架，该框架训练SLM模拟一个强大的大型LLM。因此，我们采用查询执行计划作为正式蓝图来推导这种结构化推理。我们的SLM在无结构CoT蒸馏基线上实现了8.1%的绝对提升。详细的错误分析表明，这一收益的关键因素是语法错误的显著减少。这证明了教模型使用结构化逻辑蓝图进行推理对SLMs中的可靠SQL生成是有益的。
英文摘要：
Deploying accurate Text-to-SQL systems at the enterprise level faces a difficult trilemma involving cost, security and performance. Current solutions force enterprises to choose between expensive, proprietary Large Language Models (LLMs) and low-performing Small Language Models (SLMs). Efforts to improve SLMs often rely on distilling reasoning from large LLMs using unstructured Chain-of-Thought (CoT) traces, a process that remains inherently ambiguous. Instead, we hypothesize that a formal, structured reasoning representation provides a clearer, more reliable teaching signal, as the Text-to-SQL task requires explicit and precise logical steps. To evaluate this hypothesis, we propose Struct-SQL, a novel Knowledge Distillation (KD) framework that trains an SLM to emulate a powerful large LLM. Consequently, we adopt a query execution plan as a formal blueprint to derive this structured reasoning. Our SLM, distilled with structured CoT, achieves an absolute improvement of 8.1% over an unstructured CoT distillation baseline. A detailed error analysis reveals that a key factor in this gain is a marked reduction in syntactic errors. This demonstrates that teaching a model to reason using a structured logical blueprint is beneficial for reliable SQL generation in SLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
