IndicDB -- 评估印度语言多语言文本到SQL能力的基准测试
IndicDB -- Benchmarking Multilingual Text-to-SQL Capabilities in Indian Languages
AI总结：
IndicDB是一个多语言文本到SQL基准测试，用于评估跨语言语义解析能力，包含20个数据库和237张表，通过迭代三代理框架生成15617个任务，揭示了印度语言在SQL解析中的性能差距。
摘要：
尽管大型语言模型（LLMs）显著提升了文本到SQL的性能，现有基准测试主要集中在西方环境和简化模式上，缺乏对真实世界非西方应用的评估。我们提出了IndicDB，一个多语言文本到SQL基准测试，用于评估跨语言语义解析能力。关系模式来源于开放数据平台，包括国家数据与分析平台（NDAP）和印度数据门户（IDP），确保了现实的行政数据复杂性。IndicDB包含20个数据库和237张表。为了将非规范化的政府数据转换为丰富的关系结构，我们采用迭代的三代理框架（架构师、审计员、细化器）以确保结构严谨性和高关系密度（每个数据库11.85张表；连接深度最高达六）。我们的管道是价值感知、难度校准和连接强制的，生成了15617个任务，涵盖英语、印地语和五种印度语言。我们评估了最先进的模型（DeepSeek v3.2、MiniMax 2.7、LLaMA 3.3、Qwen3）在七种语言变体上的跨语言语义解析性能。结果显示，从英语到印度语言的性能下降了9.00%，揭示了由更复杂的模式链接、增加的结构歧义和有限的外部知识驱动的“印度差距”。IndicDB为多语言文本到SQL提供了一个严谨的基准测试。代码和数据：https://anonymous.4open.science/r/multilingualText2Sql-Indic--DDCC/
英文摘要：
While Large Language Models (LLMs) have significantly advanced Text-to-SQL performance, existing benchmarks predominantly focus on Western contexts and simplified schemas, leaving a gap in real-world, non-Western applications. We present IndicDB, a multilingual Text-to-SQL benchmark for evaluating cross-lingual semantic parsing across diverse Indic languages. The relational schemas are sourced from open-data platforms, including the National Data and Analytics Platform (NDAP) and the India Data Portal (IDP), ensuring realistic administrative data complexity. IndicDB comprises 20 databases across 237 tables. To convert denormalized government data into rich relational structures, we employ an iterative three-agent framework (Architect, Auditor, Refiner) to ensure structural rigor and high relational density (11.85 tables per database; join depths up to six). Our pipeline is value-aware, difficulty-calibrated, and join-enforced, generating 15,617 tasks across English, Hindi, and five Indic languages. We evaluate cross-lingual semantic parsing performance of state-of-the-art models (DeepSeek v3.2, MiniMax 2.7, LLaMA 3.3, Qwen3) across seven linguistic variants. Results show a 9.00% performance drop from English to Indic languages, revealing an "Indic Gap" driven by harder schema linking, increased structural ambiguity, and limited external knowledge. IndicDB serves as a rigorous benchmark for multilingual Text-to-SQL. Code and data: https://anonymous.4open.science/r/multilingualText2Sql-Indic--DDCC/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
