CodeTool：通过过程监督增强大语言模型的程序化工具调用
CodeTool: Enhancing Programmatic Tool Invocation of LLMs via Process Supervision
AI总结：
针对LLM工具调用中推理路径冗长且中间步骤难以验证的问题，提出CodeTool框架，利用代码的简洁易验证性，通过即时奖励和潜在奖励两种过程奖励引导高效准确的推理，实验验证其优越性。
摘要：
工具调用显著增强了大语言模型（LLMs）的能力，然而挑战依然存在，尤其是在复杂任务场景中。当前的方法，如指令增强推理和监督微调，往往导致不必要的冗长推理路径，并且在验证中间步骤的正确性方面面临困难。在本文中，我们提出了CodeTool，一种新颖的逐步代码生成框架，通过利用代码的简洁性和易验证性来改进LLM的工具调用。CodeTool引入了两种不同的过程奖励：即时奖励（On-the-spot Reward），它为每次工具调用的准确性提供即时反馈；以及潜在奖励（Latent Reward），它评估每一步对整体任务完成的贡献。通过在每一步最大化即时奖励和潜在奖励的累积奖励，LLM被引导遵循高效且准确的推理路径。在StableToolBench和RestBench-TMDB上的大量实验证明了CodeTool相对于现有方法的优越性。
英文摘要：
Tool invocation significantly enhances the capabilities of Large Language Models (LLMs), yet challenges persist, particularly in complex task scenarios. Current methods, such as instruction-enhanced reasoning and supervised fine-tuning, often result in unnecessarily long reasoning paths and face difficulties in verifying the correctness of intermediate steps. In this paper, we propose CodeTool, a novel framework for stepwise code generation that improves LLM tool invocation by leveraging the concise and easily verifiable nature of code. CodeTool incorporates two distinct process rewards: the On-the-spot Reward, which provides immediate feedback on the accuracy of each tool invocation, and the Latent Reward, which assesses the contribution of each step toward overall task completion. By maximizing the cumulative reward of the On-the-spot and Latend Rewards at each step, LLMs are guided to follow efficient and accurate reasoning paths. Extensive experiments on StableToolBench and RestBench-TMDB demonstrate the superiority of CodeTool over existing approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
