世界建模提升语言模型智能体
World Modelling Improves Language Model Agents
AI总结：
本文提出DyMo方法，通过为LLMs增加内部状态预测能力并结合自验证采样，显著提升工具使用的成功率与可靠性，减少幻觉，为无需反复查询环境的可扩展规划RL方法开辟路径。
摘要：
在有状态环境中使用工具给大语言模型（LLMs）带来了独特挑战，现有依赖在环境中重复试错的测试时计算策略并不实用。我们提出动力学建模（DyMo），一种在后训练阶段在函数调用之外为LLMs增加状态预测能力的方法。这使得LLMs能够通过内部环境模型预测其动作的未来状态。在Berkeley Function Calling Leaderboard V2上，DyMo提高了成功率并显著减少了幻觉。我们进一步将内部环境模型整合到自验证采样（SVS）中，并表明这大幅提升了在试验次数k上的pass^k指标，同时使模型能够拒绝不可靠的输出。DyMo与SVS共同大幅增强了LLMs在工具使用中的有效性和可靠性。我们相信这项工作为无需反复查询真实环境即可实现可扩展的LLM推理规划强化学习方法指明了一条道路。
英文摘要：
Tool use in stateful environments presents unique challenges for large language models (LLMs), where existing test-time compute strategies relying on repeated trials in the environment are impractical. We propose dynamics modelling (DyMo), a method that augments LLMs with a state prediction capability alongside function calling during post-training. This enables LLMs to predict the future states of their actions through an internal environment model. On the Berkeley Function Calling Leaderboard V2, DyMo improves success rates and significantly reduces hallucinations. We further integrate the internal environment model into self-verification sampling (SVS), and show that this substantially improves pass^k over number of trials k, and allows the model to refuse unreliable outputs. Together, DyMo and SVS greatly enhance the effectiveness and reliability of LLMs for tool use. We believe this work charts a path towards scalable planning RL methods for LLM inference without repeatedly querying the oracle environment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
