评估用于基于自然语言的SQL和API调用生成的检索增强生成变体
RAG Strategies for Natural Language-Based SQL Query and REST API Call Generation
- IU International University of Applied Sciences（国际应用科学大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文评估了三种RAG变体在生成SQL和API调用任务中的性能，发现CoRAG在混合文档环境下表现最佳，检索策略设计对自然语言接口至关重要。
AI中文摘要：
企业系统日益需要能够将用户请求转换为结构化操作（如SQL查询和REST API调用）的自然语言接口。尽管大语言模型（LLMs）在代码生成方面显示出潜力[Chen等，2021；Huynh和Lin，2025]，但在特定领域的企业环境中其有效性仍待探索，特别是在需要同时处理检索和修改任务的情况下。本文全面评估了三种检索增强生成（RAG）变体[Lewis等，2021]——标准RAG、Self-RAG[Asai等，2024]和CoRAG[Wang等，2025]——在SQL查询生成、REST API调用生成以及需要动态任务分类的综合任务中的表现。使用SAP交易银行作为现实的企业用例，我们构建了一个涵盖两种模态的新型测试数据集，并在数据库-only、API-only和混合文档上下文中评估了18种实验配置。结果表明，RAG是必不可少的：没有检索时，所有任务的精确匹配准确率均为0%，而检索带来了显著的执行准确率提升（最高79.30%）和组件匹配准确率提升（最高78.86%）。关键的是，CoRAG在混合文档设置中表现最稳健，其在综合任务中的精确匹配准确率（10.29%）比标准RAG（7.45%）有统计学显著提升，主要得益于更出色的SQL生成性能（15.32% vs. 11.56%）。我们的发现确立了检索策略设计作为生产级自然语言接口的关键决定因素，显示在文档异质性下，迭代查询分解优于top-k检索和二元相关性过滤。
英文摘要：
Enterprise software systems commonly expose business functionality through both relational databases and REST APIs. Accessing these interfaces requires specialized technical knowledge, as users must determine whether a request requires a database query or an API operation and understand the corresponding schemas, endpoints, and parameters. This creates demand for natural language interfaces that translate user requests into SQL queries and REST API calls. While large language models (LLMs) show promise for structured code generation, they typically lack reliable knowledge of enterprise-specific schemas, endpoints, and documentation. Retrieval-augmented generation (RAG) addresses this limitation by grounding generation in external documentation. However, prior work largely studies SQL query generation and REST API call generation separately, despite enterprise documentation environments often containing both database schemas and API specifications. We systematically evaluate standard RAG, Self-RAG, and CoRAG across SQL query generation, REST API call generation, and a combined task requiring routing between both operation types. Using SAP Transactional Banking as a realistic enterprise use case, we constructed an execution-validated dataset and compared retrieval strategies under database-only, API-only, and mixed-documentation settings. Retrieval augmentation proved essential for reliable enterprise structured generation, substantially improving performance over a no-retrieval baseline. CoRAG achieved the best results in the combined SQL query and REST API call setting, with statistically significant improvements in exact-match accuracy over standard RAG, primarily driven by stronger SQL query generation under mixed-documentation retrieval conditions. Overall, findings show that retrieval strategy substantially affects structured generation performance under mixed-documentation settings.
