WALL-E：通过规则学习实现的世界对齐改进基于世界模型的LLM智能体
WALL-E: World Alignment by Rule Learning Improves World Model-based LLM Agents
AI总结：
本文提出神经符号方法WALL-E，通过无梯度规则学习将LLM与环境动态对齐以构建精确世界模型，并基于MPC优化前瞻动作，在Minecraft和ALFWorld中以更低成本取得更高成功率。
摘要：
大型语言模型（LLM）能否直接作为基于模型的智能体的强大世界模型？尽管LLM的先验知识与特定环境动态之间存在差距，但我们的研究表明，这种差距可以通过将LLM与其部署环境对齐来弥合，而这种“世界对齐”可以通过在LLM上进行规则学习来高效实现。鉴于LLM具备丰富的先验知识，仅需少量额外规则即可使LLM的预测与特定环境动态对齐。为此，我们提出一种神经符号方法，通过LLM无梯度地学习这些规则，具体做法是根据智能体探索的轨迹与世界模型预测的比较结果来归纳、更新和剪枝规则。最终得到的世界模型由LLM和学到的规则组成。我们的具身LLM智能体“WALL-E”基于模型预测控制（MPC）构建。通过基于精确的世界模型优化前瞻动作，MPC显著提升了探索与学习效率。与现有LLM智能体相比，WALL-E的推理仅需少量主要规则，而无需在LLM输入中包含冗长的缓冲轨迹。在Minecraft和ALFWorld的开放世界挑战中，WALL-E取得了比现有方法更高的成功率，且在重规划时间和推理所用token数量上成本更低。在Minecraft中，WALL-E的成功率比基线高出15-30%，同时重规划轮数减少8-20轮，token消耗仅为60-80%。在ALFWorld中，仅需6次迭代，其成功率便飙升至95%的新纪录。
英文摘要：
Can large language models (LLMs) directly serve as powerful world models for model-based agents? While the gaps between the prior knowledge of LLMs and the specified environment's dynamics do exist, our study reveals that the gaps can be bridged by aligning an LLM with its deployed environment and such "world alignment" can be efficiently achieved by rule learning on LLMs. Given the rich prior knowledge of LLMs, only a few additional rules suffice to align LLM predictions with the specified environment dynamics. To this end, we propose a neurosymbolic approach to learn these rules gradient-free through LLMs, by inducing, updating, and pruning rules based on comparisons of agent-explored trajectories and world model predictions. The resulting world model is composed of the LLM and the learned rules. Our embodied LLM agent "WALL-E" is built upon model-predictive control (MPC). By optimizing look-ahead actions based on the precise world model, MPC significantly improves exploration and learning efficiency. Compared to existing LLM agents, WALL-E's reasoning only requires a few principal rules rather than verbose buffered trajectories being included in the LLM input. On open-world challenges in Minecraft and ALFWorld, WALL-E achieves higher success rates than existing methods, with lower costs on replanning time and the number of tokens used for reasoning. In Minecraft, WALL-E exceeds baselines by 15-30% in success rate while costing 8-20 fewer replanning rounds and only 60-80% of tokens. In ALFWorld, its success rate surges to a new record high of 95% only after 6 iterations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
