揭示语言智能体在规划中的障碍
Revealing the Barriers of Language Agents in Planning
AI总结:
本文通过特征归因研究揭示了阻碍语言智能体规划的两个关键因素:约束作用有限和问题影响减弱,并指出当前策略仅能缓解而无法完全解决这些挑战。
摘要:
自人工智能诞生以来,自主规划一直是一个持续追求的目标。基于精心设计的求解器,早期的规划智能体能够为特定任务提供精确的解决方案,但缺乏泛化能力。大型语言模型(LLMs)的出现及其强大的推理能力,通过自动为给定任务生成合理的解决方案,重新激发了人们对自主规划的兴趣。然而,先前的研究和我们的实验表明,当前的语言智能体仍然缺乏人类水平的规划能力。即使是最先进的推理模型 OpenAI o1,在一个复杂的真实世界规划基准测试上也仅取得了 15.6% 的成绩。这凸显了一个关键问题:是什么阻碍了语言智能体达到人类水平的规划能力?尽管现有研究已经指出了智能体规划方面的不佳表现,但更深层次的潜在问题,以及为解决这些问题而提出的策略的机制和局限性,仍未得到充分理解。在这项工作中,我们应用了特征归因研究,并识别出阻碍智能体规划的两个关键因素:约束的有限作用和问题影响的减弱。我们还发现,尽管当前策略有助于缓解这些挑战,但并不能完全解决它们,这表明智能体在达到人类水平智能之前仍有很长的路要走。
英文摘要:
Autonomous planning has been an ongoing pursuit since the inception of artificial intelligence. Based on curated problem solvers, early planning agents could deliver precise solutions for specific tasks but lacked generalization. The emergence of large language models (LLMs) and their powerful reasoning capabilities has reignited interest in autonomous planning by automatically generating reasonable solutions for given tasks. However, prior research and our experiments show that current language agents still lack human-level planning abilities. Even the state-of-the-art reasoning model, OpenAI o1, achieves only 15.6% on one of the complex real-world planning benchmarks. This highlights a critical question: What hinders language agents from achieving human-level planning? Although existing studies have highlighted weak performance in agent planning, the deeper underlying issues and the mechanisms and limitations of the strategies proposed to address them remain insufficiently understood. In this work, we apply the feature attribution study and identify two key factors that hinder agent planning: the limited role of constraints and the diminishing influence of questions. We also find that although current strategies help mitigate these challenges, they do not fully resolve them, indicating that agents still have a long way to go before reaching human-level intelligence.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。