arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2511.09148cs.CLcs.AIcs.LG
收藏

LoopTool:面向鲁棒大语言模型工具调用的闭环数据-训练框架

LoopTool: Closing the Data-Training Loop for Robust LLM Tool Calls

Shanghai Jiao Tong University(上海交通大学);Xiaohongshu Inc.(小红书公司)

Kangning Zhang, Wenxiang Jiao, Kounianhua Du, Yuan Lu, Weiwen Liu, Weinan Zhang, Yong Yu

AI总结:

针对静态工具学习数据管线割裂、效率低的问题,提出闭环框架LoopTool,通过三个协同模块迭代优化数据与模型,训练的8B模型性能超越32B生成器,在相关基准取得同规模最优结果。

摘要:

为大语言模型(LLM)接入外部工具可使其执行复杂多步任务,但当前工具学习受静态合成数据管线制约:数据生成与模型训练是两个独立、无交互的流程,既无法自适应聚焦模型的特定弱点,还会让噪声标签持续存在,降低训练效率。本文提出LoopTool,这是一个全自动化、模型感知的数据演化框架,通过紧密整合数据合成与模型训练打通上述流程。LoopTool借助三个协同模块迭代优化数据与模型:一是贪心能力探测(GCP)模块,诊断模型已掌握和未掌握的能力;二是判断引导标签验证(JGLV)模块,使用开源评判模型发现并修正标注错误,逐步净化数据集;三是错误驱动数据扩展(EDDE)模块,基于识别出的失败案例生成有挑战性的新样本。该闭环流程运行在高性价比的开源生态中,无需依赖昂贵的闭源API。实验显示,经LoopTool训练的8B参数模型性能显著超越作为其数据生成器的32B参数模型,在BFCL-v3和ACEBench基准上取得了同规模下的当前最优结果。本研究证明,闭环自优化数据管线可大幅提升大语言模型的工具使用能力。

英文摘要:

Augmenting Large Language Models (LLMs) with external tools enables them to execute complex, multi-step tasks. However, tool learning is hampered by the static synthetic data pipelines where data generation and model training are executed as two separate, non-interactive processes. This approach fails to adaptively focus on a model's specific weaknesses and allows noisy labels to persist, degrading training efficiency. We introduce LoopTool, a fully automated, model-aware data evolution framework that closes this loop by tightly integrating data synthesis and model training. LoopTool iteratively refines both the data and the model through three synergistic modules: (1) Greedy Capability Probing (GCP) diagnoses the model's mastered and failed capabilities; (2) Judgement-Guided Label Verification (JGLV) uses an open-source judge model to find and correct annotation errors, progressively purifying the dataset; and (3) Error-Driven Data Expansion (EDDE) generates new, challenging samples based on identified failures. This closed-loop process operates within a cost-effective, open-source ecosystem, eliminating dependence on expensive closed-source APIs. Experiments show that our 8B model trained with LoopTool significantly surpasses its 32B data generator and achieves new state-of-the-art results on the BFCL-v3 and ACEBench benchmarks for its scale. Our work demonstrates that closed-loop, self-refining data pipelines can dramatically enhance the tool-use capabilities of LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑