在草莓田中规划:评估与改进 LRM o1 的规划与调度能力
Planning in Strawberry Fields: Evaluating and Improving the Planning and Scheduling Capabilities of LRM o1
AI总结:
本文评估了 OpenAI o1 系列大型推理模型在规划与调度任务上的能力,发现其相比自回归 LLM 有显著提升但推理成本高且缺乏正确性保证,并提出结合外部验证器的 LRM-Modulo 系统以同时保证输出正确性并提升性能。
摘要:
长期以来,规划一系列行动以实现期望状态的能力一直被视为智能体的核心能力,并且自人工智能研究诞生以来就是其不可分割的组成部分。随着大语言模型(LLMs)的出现,人们对其是否具备此类规划能力产生了相当大的兴趣,但尽管自 GPT3 以来涌现了大量新的私有和开源 LLM,进展仍然缓慢。OpenAI 声称其最近的 o1(Strawberry)模型经过专门构建和训练,以摆脱自回归 LLM 的常规限制——使其成为一种新型模型:大型推理模型(LRM)。在本文中,我们在规划和调度基准上评估了两种 LRM(o1-preview 和 o1-mini)的规划能力。我们发现,虽然 o1 似乎确实比自回归 LLM 提供了显著改进,但这是以高昂的推理成本为代价的,同时仍未能为其生成的内容提供任何保证。我们还表明,将 o1 模型与外部验证器结合——在所谓的 LRM-Modulo 系统中——可以保证组合系统输出的正确性,同时进一步提高性能。
英文摘要:
The ability to plan a course of action that achieves a desired state of affairs has long been considered a core competence of intelligent agents and has been an integral part of AI research since its inception. With the advent of large language models (LLMs), there has been considerable interest in the question of whether or not they possess such planning abilities, but -- despite the slew of new private and open source LLMs since GPT3 -- progress has remained slow. OpenAI claims that their recent o1 (Strawberry) model has been specifically constructed and trained to escape the normal limitations of autoregressive LLMs -- making it a new kind of model: a Large Reasoning Model (LRM). In this paper, we evaluate the planning capabilities of two LRMs (o1-preview and o1-mini) on both planning and scheduling benchmarks. We see that while o1 does seem to offer significant improvements over autoregressive LLMs, this comes at a steep inference cost, while still failing to provide any guarantees over what it generates. We also show that combining o1 models with external verifiers -- in a so-called LRM-Modulo system -- guarantees the correctness of the combined system's output while further improving performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。