arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.28601cs.DBcs.AI
收藏

面向真实世界大型数据库的文本到SQL数据库上下文压缩

Database Context Compression for Text-to-SQL on Real-World Large Databases

National Engineering Research Center of Software Engineering, Peking University(软件工程国家工程研究中心,北京大学);School of Computer Science, Peking University(北京大学计算机学院);Peking University Information Technology Institute, Tianjin Binhai(天津滨海大学信息技术研究所,北京大学)

Jingwen Liu, Weibin Liao, Xin Gao, Junfeng Zhao, Yasha Wang

AI总结:

针对真实大型数据库中文本到SQL性能瓶颈,提出基于SGCF原则的数据库上下文压缩方法DBCC,通过离线结构语义压缩和在线证据净化,将输入上下文减少两个数量级,并提升模式链接召回率和执行准确率。

摘要:

文本到SQL的最新进展得益于更强的语言模型和提示策略,但在Spider 2.0和BIRD等真实企业基准上的表现仍远低于经典学术数据集。我们认为主要瓶颈不再是推理,而是数据库表示。真实数据库包含重复的审计列、大量相似表、含义仅存储在文档中的不透明标识符,以及包含大量与查询无关信息的数据字典。现有的查询感知方法(包括模式链接和基于检索的模式选择)过滤了这些原始上下文,但仍操作在冗余和冗长的表示上。我们将问题重新表述为数据库上下文压缩,这是一种查询无关的转换,将模式、语义描述和外部文档重写为紧凑表示。我们使用SGCF(支持-增益组件分解)原则形式化该转换,该原则在单一覆盖目标下统一了重复列提取、同构表模板化、语义组件化和证据净化。基于SGCF,我们提出DBCC,一种数据库端中间件,执行离线结构和语义压缩以及轻量级在线证据净化。DBCC是模型无关的,可集成到现有文本到SQL流水线中。在Spider 2.0-Snow和BIRD上,DBCC将输入上下文减少多达两个数量级(在最大的Spider 2.0-Snow子集上从2.6M token降至34.7K token),在DeepSeek-V3.2下将模式链接严格召回率从0%提升至56.5%(在Claude Opus 4.7下为63.1%),并在三个近期文本到SQL系统上持续将端到端执行准确率提高1.8-1.9%。我们的代码已在https://github.com/MrBlankness/SchemaCompression开源。

英文摘要:

Recent progress in Text-to-SQL has been driven by stronger language models and prompting strategies, yet performance on real enterprise benchmarks such as Spider 2.0 and BIRD remains far below that on classical academic datasets. We argue that the main bottleneck is no longer reasoning, but database representation. Real databases contain repeated audit columns, large groups of similar tables, opaque identifiers whose meanings are stored only in documentation, and extensive data dictionaries with little query-relevant information. Existing query-aware methods, including schema linking and retrieval-based schema selection, filter this raw context but still operate on redundant and verbose representations. We reformulate the problem as database context compression, a query-agnostic transformation that rewrites schemas, semantic descriptions, and external documentation into a compact representation. We formalize this transformation with the SGCF (Support-Gain Component Factorization) principle, which unifies repeated column extraction, isomorphic table templating, semantic componentization, and evidence purification under a single coverage objective. Based on SGCF, we propose DBCC, a database-side middleware that performs offline structural and semantic compression together with lightweight online evidence purification. DBCC is model-agnostic and can be integrated into existing Text-to-SQL pipelines. On Spider 2.0-Snow and BIRD, DBCC reduces input context by up to two orders of magnitude (from 2.6M to 34.7K tokens on the largest Spider 2.0-Snow subset), improves schema-linking strict recall from 0% to 56.5% under DeepSeek-V3.2 (63.1% under Claude Opus 4.7), and consistently increases end-to-end execution accuracy by 1.8-1.9% over three recent Text-to-SQL systems. Our code is open-sourced at https://github.com/MrBlankness/SchemaCompression.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑