对《思考的幻觉》的评论：将推理悬崖重新定义为智能体差距
A Comment On "The Illusion of Thinking": Reframing the Reasoning Cliff as an Agentic Gap
AI总结：
本文反驳“推理悬崖”为内在局限，指出其源于静态文本评估的系统约束，并以智能体差距重新定义，实证展示工具使模型超越原失败阈值。
摘要：
Shojaee等人（2025）的近期工作《思考的幻觉：通过问题复杂性的视角理解推理模型的优势与局限》呈现了一个引人注目的实证发现，即“推理悬崖”：大型推理模型（LRMs）的性能在超过特定复杂度阈值后急剧崩溃，作者将此归因于思维链（Chain-of-Thought, CoT）推理的内在扩展限制。本评论在认可该研究方法严谨性的同时，认为这一结论受到实验伪影的混淆。我们主张，观察到的失败并非基本认知边界的证据，而是静态、纯文本评估范式中系统级约束的可预测结果，这些约束包括工具使用限制、上下文窗口回忆问题、缺乏关键认知基线、统计报告不足以及输出生成限制。我们通过“智能体差距”的视角重新定义这一性能崩溃，断言模型并非在推理上失败，而是在一个极度受限的界面内执行失败。我们通过展示一个惊人的反转来实证支持这一批评：一个最初在仅限文本生成时宣称谜题无法解决的模型，现在借助智能体工具不仅解决了该谜题，还掌握了远超其先前未能跨越的推理悬崖的复杂度变体。此外，我们对o4-mini和GPT-4o等支持工具的模型的实证分析揭示了智能体推理的层级结构，从简单程序执行到复杂元认知自我修正，这对我们定义和衡量机器智能具有重要启示。归因于LRMs的“思考的幻觉”与其说是推理缺陷，不如说是一个本有能力的心灵缺乏行动工具的结果。
英文摘要：
The recent work by Shojaee et al. (2025), titled The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity, presents a compelling empirical finding, a reasoning cliff, where the performance of Large Reasoning Models (LRMs) collapses beyond a specific complexity threshold, which the authors posit as an intrinsic scaling limitation of Chain-of-Thought (CoT) reasoning. This commentary, while acknowledging the study's methodological rigor, contends that this conclusion is confounded by experimental artifacts. We argue that the observed failure is not evidence of a fundamental cognitive boundary, but rather a predictable outcome of system-level constraints in the static, text-only evaluation paradigm, including tool use restrictions, context window recall issues, the absence of crucial cognitive baselines, inadequate statistical reporting, and output generation limits. We reframe this performance collapse through the lens of an agentic gap, asserting that the models are not failing at reasoning, but at execution within a profoundly restrictive interface. We empirically substantiate this critique by demonstrating a striking reversal. A model, initially declaring a puzzle impossible when confined to text-only generation, now employs agentic tools to not only solve it but also master variations of complexity far beyond the reasoning cliff it previously failed to surmount. Additionally, our empirical analysis of tool-enabled models like o4-mini and GPT-4o reveals a hierarchy of agentic reasoning, from simple procedural execution to complex meta-cognitive self-correction, which has significant implications for how we define and measure machine intelligence. The illusion of thinking attributed to LRMs is less a reasoning deficit and more a consequence of an otherwise capable mind lacking the tools for action.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
