大语言模型的推理是潜在的,而非思维链
LLM Reasoning Is Latent, Not the Chain of Thought
AI总结:
本文提出大语言模型推理应研究为潜在状态轨迹形成而非表面思维链。研究指出,推理对象的定义影响了忠实性、可解释性、推理基准和推理时干预的主张。通过分离三个混淆因素,提出三种假设,发现当前证据更支持潜在状态轨迹作为默认假设。
摘要:
本文立场论文主张,大语言模型(LLM)推理应研究为潜在状态轨迹形成,而非忠实的表面思维链(CoT)。这很重要,因为关于忠实性、可解释性、推理基准和推理时干预的主张都依赖于该领域认为推理的主要对象是什么。我们询问在分离三个常被混淆的因素后,该对象应是什么,并正式化三种竞争假设:H1,推理主要由潜在状态轨迹中介;H2,推理主要由显式表面CoT中介;H0,大多数明显推理收益更可能由通用序列计算而非任何特权表征对象解释。通过将最近的实证、机理和调查工作重新组织在此框架下,并添加经过计算审计的工作示例,这些示例分解了表面痕迹、潜在干预和匹配预算扩展,我们发现当前证据最有力地支持H1作为默认工作假设,而非任务无关的结论。因此,我们提出两个建议:该领域应将潜在状态动态作为LLM推理的默认研究对象,并应通过设计明确分离表面痕迹、潜在状态和序列计算来评估推理。
英文摘要:
This position paper argues that large language model (LLM) reasoning should be studied as latent-state trajectory formation rather than as faithful surface chain-of-thought (CoT). This matters because claims about faithfulness, interpretability, reasoning benchmarks, and inference-time intervention all depend on what the field takes the primary object of reasoning to be. We ask what that object should be once three often-confounded factors are separated and formalize three competing hypotheses: H1, reasoning is primarily mediated by latent-state trajectories; H2, reasoning is primarily mediated by explicit surface CoT; and H0, most apparent reasoning gains are better explained by generic serial compute than by any privileged representational object. Reorganizing recent empirical, mechanistic, and survey work under this framework, and adding compute-audited worked exemplars that factorize surface traces, latent interventions, and matched budget expansions, we find that current evidence most strongly supports H1 as a default working hypothesis rather than as a task-independent verdict. We therefore make two recommendations: the field should treat latent-state dynamics as the default object of study for LLM reasoning, and it should evaluate reasoning with designs that explicitly disentangle surface traces, latent states, and serial compute.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。