以消歧为核心的知识蒸馏使企业工具调用LLM更真实且更安全
Disambiguation-Centric Finetuning Makes Enterprise Tool-Calling LLMs More Realistic and Less Risky
AI总结：
本文提出DiaFORGE框架，通过三阶段流程提升企业工具调用LLM的准确性和安全性，实验显示其在工具调用成功率上优于GPT-4o和Claude-3.5-Sonnet。
摘要：
大型语言模型（LLMs）越来越多地被要求调用企业API，但它们在面对近似工具竞争或必要参数未明确时常常失败。我们引入DiaFORGE（对话框架用于有机响应生成与评估），一个以消歧为核心、三阶段流程，包括（i）合成驱动个性的多轮对话，在其中助手必须区分高度相似的工具；（ii）对开源模型进行监督微调，参数范围为3B-70B；（iii）通过动态套件评估现实准备度，重新部署每个模型在实时代理循环中，并报告端到端目标完成情况以及传统静态指标。在我们的动态基准DiaBENCH上，使用DiaFORGE训练的模型在优化提示下，工具调用成功率比GPT-4o高27个百分点，比Claude-3.5-Sonnet高49个百分点。为促进进一步研究，我们发布了一个包含5000个生产级企业API规范的开放语料库，配以严格验证的消歧对话，提供构建可靠、企业级工具调用代理的实用蓝图。
英文摘要：
Large language models (LLMs) are increasingly tasked with invoking enterprise APIs, yet they routinely falter when near-duplicate tools vie for the same user intent or when required arguments are left underspecified. We introduce DiaFORGE (Dialogue Framework for Organic Response Generation & Evaluation), a disambiguation-centric, three-stage pipeline that (i) synthesizes persona-driven, multi-turn dialogues in which the assistant must distinguish among highly similar tools, (ii) performs supervised fine-tuning of open-source models with reasoning traces across 3B - 70B parameters, and (iii) evaluates real-world readiness via a dynamic suite that redeploys each model in a live agentic loop and reports end-to-end goal completion alongside conventional static metrics. On our dynamic benchmark DiaBENCH, models trained with DiaFORGE raise tool-invocation success by 27 pp over GPT-4o and by 49 pp over Claude-3.5-Sonnet, both under optimized prompting. To spur further research, we release an open corpus of 5000 production-grade enterprise API specifications paired with rigorously validated, disambiguation-focused dialogues, offering a practical blueprint for building reliable, enterprise-ready tool-calling agents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
