思考以回忆：推理如何解锁大语言模型中的参数知识
Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs
AI总结：
本研究发现推理通过计算缓冲效应和事实优先机制提升大语言模型对参数知识的回忆能力，但可能增加幻觉风险，通过优化推理轨迹可提高模型准确性。
摘要：
尽管在大语言模型（LLMs）中推理在数学、代码生成和多跳事实性问题中扮演着自然角色，但其对简单、单跳事实性问题的影响仍不明确。此类问题不需要分步骤的逻辑分解，使得推理的效用显得高度反直觉。然而，我们发现启用推理显著扩展了模型参数知识回忆的能力边界，解锁了原本无法达到的正确答案。为什么在没有复杂推理步骤的情况下，推理会帮助参数知识回忆？为了解答这个问题，我们设计了一系列假设驱动的受控实验，并识别出两个关键驱动机制：（1）计算缓冲效应，即模型利用生成的推理标记进行与语义内容无关的潜在计算；（2）事实优先，即生成相关主题的事实起到语义桥梁的作用，促进正确答案的检索。重要的是，后者生成性自我检索机制本身存在固有风险：我们证明在推理过程中生成中间事实会增加最终答案中幻觉的可能性。最后，我们展示我们的见解可以被用来直接提高模型的准确性，通过优先选择包含无幻觉事实陈述的推理轨迹。
英文摘要：
While reasoning in LLMs plays a natural role in math, code generation, and multi-hop factual questions, its effect on simple, single-hop factual questions remains unclear. Such questions do not require step-by-step logical decomposition, making the utility of reasoning highly counterintuitive. Nevertheless, we find that enabling reasoning substantially expands the capability boundary of the model's parametric knowledge recall, unlocking correct answers that are otherwise effectively unreachable. Why does reasoning aid parametric knowledge recall when there are no complex reasoning steps to be done? To answer this, we design a series of hypothesis-driven controlled experiments, and identify two key driving mechanisms: (1) a computational buffer effect, where the model uses the generated reasoning tokens to perform latent computation independent of their semantic content; and (2) factual priming, where generating topically related facts acts as a semantic bridge that facilitates correct answer retrieval. Importantly, this latter generative self-retrieval mechanism carries inherent risks: we demonstrate that hallucinating intermediate facts during reasoning increases the likelihood of hallucinations in the final answer. Finally, we show that our insights can be harnessed to directly improve model accuracy by prioritizing reasoning trajectories that contain hallucination-free factual statements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
