BIRDTurk: BIRD文本到SQL数据集对土耳其语的适应
BIRDTurk: Adaptation of the BIRD Text-to-SQL Dataset to Turkish
AI总结：
BIRDTurk是首个将BIRD文本到SQL基准适应到土耳其语的数据集，通过受控翻译流程保持SQL逻辑结构，评估了代理推理和监督微调在土耳其语中的表现，揭示了语言差异对性能的影响。
摘要：
文本到SQL系统在英语基准测试中表现强劲，但其在形态丰富、低资源语言中的行为仍鲜有研究。我们介绍了BIRDTurk，这是首个将BIRD基准适应到土耳其语的数据集，通过受控的翻译流程构建，将模式标识符适应到土耳其语，同时严格保持SQL查询和数据库的逻辑结构和执行语义。翻译质量在通过中心极限定理确定的样本量上得到验证，实现了98.15%的人工评估样本准确率。使用BIRDTurk，我们评估了基于推理的提示、代理多阶段推理和监督微调。我们的结果表明，土耳其语引入了持续的性能下降，这由结构语言差异和在LLM预训练中的代表性不足驱动，而代理推理在跨语言鲁棒性方面表现更强。监督微调对标准多语言基线仍然具有挑战性，但能与现代指令调优模型有效扩展。BIRDTurk为在现实数据库条件下进行跨语言文本到SQL评估提供了受控的测试环境。我们释放了训练和开发分割以支持未来研究。
英文摘要：
Text-to-SQL systems have achieved strong performance on English benchmarks, yet their behavior in morphologically rich, low-resource languages remains largely unexplored. We introduce BIRDTurk, the first Turkish adaptation of the BIRD benchmark, constructed through a controlled translation pipeline that adapts schema identifiers to Turkish while strictly preserving the logical structure and execution semantics of SQL queries and databases. Translation quality is validated on a sample size determined by the Central Limit Theorem to ensure 95% confidence, achieving 98.15% accuracy on human-evaluated samples. Using BIRDTurk, we evaluate inference-based prompting, agentic multi-stage reasoning, and supervised fine-tuning. Our results reveal that Turkish introduces consistent performance degradation, driven by both structural linguistic divergence and underrepresentation in LLM pretraining, while agentic reasoning demonstrates stronger cross-lingual robustness. Supervised fine-tuning remains challenging for standard multilingual baselines but scales effectively with modern instruction-tuned models. BIRDTurk provides a controlled testbed for cross-lingual Text-to-SQL evaluation under realistic database conditions. We release the training and development splits to support future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
