通过认知工具激发语言模型的推理能力
Eliciting Reasoning in Language Models with Cognitive Tools
AI总结：
通过引入认知工具，提升语言模型在数学推理任务上的性能，探索推理机制并补充训练后方法的作用。
摘要：
最近，像OpenAI的o1这样的推理模型的出现，引发了人工智能界对这些封闭模型背后机制的热烈讨论，随后涌现出大量复制研究，尤其是开源社区的大量工作。这些推测主要通过DeepSeek-R1的演示得到解决，该演示表明链式思维和强化学习（RL）可以有效地在基础大语言模型（LLM）上复制推理。然而，探索其他理论上激发推理的方法仍然具有价值，这有助于阐明底层机制，并提供额外的方法，可能带来互补的好处。本文基于长期存在的认知心理学和认知架构文献，认为推理源于一组模块化、预设认知操作的有序、连续执行。关键的是，我们将在现代代理工具调用框架中实现这一核心思想。具体而言，我们为LLM配备了一组小的“认知工具”，这些工具封装了特定的推理操作，每个操作由LLM本身执行。令人惊讶的是，这种简单策略在标准数学推理基准测试中相对于基础LLM表现出显著的性能提升，无论是封闭模型还是开放权重模型。例如，将我们的“认知工具”提供给GPT-4.1，使其在AIME2024上的pass@1性能从32%提升到53%，甚至超过了o1-preview的性能。除了实际意义外，这一演示还参与了关于训练后方法在激发LLM推理中的作用与训练前获得的固有能力作用的辩论，并探讨了训练后是否仅仅是揭示这些潜在能力的问题。
英文摘要：
The recent advent of reasoning models like OpenAI's o1 was met with excited speculation by the AI community about the mechanisms underlying these capabilities in closed models, followed by a rush of replication efforts, particularly from the open source community. These speculations were largely settled by the demonstration from DeepSeek-R1 that chains-of-thought and reinforcement learning (RL) can effectively replicate reasoning on top of base LLMs. However, it remains valuable to explore alternative methods for theoretically eliciting reasoning that could help elucidate the underlying mechanisms, as well as providing additional methods that may offer complementary benefits. Here, we build on the long-standing literature in cognitive psychology and cognitive architectures, which postulates that reasoning arises from the orchestrated, sequential execution of a set of modular, predetermined cognitive operations. Crucially, we implement this key idea within a modern agentic tool-calling framework. In particular, we endow an LLM with a small set of "cognitive tools" encapsulating specific reasoning operations, each executed by the LLM itself. Surprisingly, this simple strategy results in considerable gains in performance on standard mathematical reasoning benchmarks compared to base LLMs, for both closed and open-weight models. For instance, providing our "cognitive tools" to GPT-4.1 increases its pass@1 performance on AIME2024 from 32% to 53%, even surpassing the performance of o1-preview. In addition to its practical implications, this demonstration contributes to the debate regarding the role of post-training methods in eliciting reasoning in LLMs versus the role of inherent capabilities acquired during pre-training, and whether post-training merely uncovers these latent abilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
