通过环境交互教授LLMs学习工具尝试与执行
Teaching LLMs to Learn Tool Trialing and Execution through Environment Interaction
AI总结：
ToolMaster通过环境交互主动学习工具使用，提升LLMs在新工具上的泛化与鲁棒性。
摘要：
给大型语言模型（LLMs）配备外部工具使它们能够解决复杂的真实世界问题。然而，现有方法在面对新出现或演变的工具时，鲁棒性仍是一个关键挑战。现有轨迹中心的范式主要依赖于训练期间记忆静态的解决方案路径，这限制了LLMs将工具使用泛化到新引入或以前未见过的工具的能力。在本文中，我们提出了ToolMaster，一个框架，将工具使用从模仿黄金工具调用轨迹转向通过与环境互动主动学习工具使用。为了优化LLMs的工具规划和调用能力，ToolMaster采用尝试与执行范式，该范式训练LLMs首先模仿包含显式工具尝试和自我纠正的教师生成轨迹，随后通过强化学习协调尝试和执行阶段。这一过程使代理能够通过主动与环境互动自主探索正确的工具使用，并形成有益于工具执行的经验知识。实验结果表明，ToolMaster在未见过或不熟悉的工具上，在泛化和鲁棒性方面显著优于现有基线。所有代码和数据均可在https://github.com/NEUIR/ToolMaster获取。
英文摘要：
Equipping Large Language Models (LLMs) with external tools enables them to solve complex real-world problems. However, the robustness of existing methods remains a critical challenge when confronting novel or evolving tools. Existing trajectory-centric paradigms primarily rely on memorizing static solution paths during training, which limits the ability of LLMs to generalize tool usage to newly introduced or previously unseen tools. In this paper, we propose ToolMaster, a framework that shifts tool use from imitating golden tool-calling trajectories to actively learning tool usage through interaction with the environment. To optimize LLMs for tool planning and invocation, ToolMaster adopts a trial-and-execution paradigm, which trains LLMs to first imitate teacher-generated trajectories containing explicit tool trials and self-correction, followed by reinforcement learning to coordinate the trial and execution phases jointly. This process enables agents to autonomously explore correct tool usage by actively interacting with environments and forming experiential knowledge that benefits tool execution. Experimental results demonstrate that ToolMaster significantly outperforms existing baselines in terms of generalization and robustness across unseen or unfamiliar tools. All code and data are available at https://github.com/NEUIR/ToolMaster.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
