从双跳潜在推理研究中获得的启示
Lessons from Studying Two-Hop Latent Reasoning
AI总结：
研究通过双跳问答任务揭示LLM潜在推理能力，发现模型在合成事实组合上表现不足，但能处理混合事实场景，强调避免误判推理能力的必要性。
摘要：
大型语言模型可以使用链式思维（CoT）来外部化推理， potentially enabling oversight of capable LLM agents. 前期工作表明，模型在没有CoT的情况下在双跳问答任务上存在困难。这种能力如此基础，如果它是基本限制，就暗示许多复杂代理任务同样需要CoT。我们通过双跳问答任务作为案例研究，调查LLM的潜在推理能力。之前关于潜在和外部化双跳推理差距的研究产生了混杂证据和不确定结果。在本文中，我们引入了一个受控的环境来研究LLM的双跳推理能力，其中正结果提供了对潜在推理的确定性证据。我们对LLM（包括Llama 3 8B和GPT-4o）进行合成事实的微调，并在这些事实上测试双跳推理。通过使用合成事实，我们排除了记忆和推理捷径作为双跳表现解释的可能性。我们观察到一幅细致的图景：模型无法组合两个合成事实，但在一个事实是合成、另一个是自然的情况下可以成功。这些结果表明LLM确实具备潜在的双跳推理能力，尽管这种能力如何随着模型规模扩展仍不清楚。最后，我们强调了一个教训：对于研究LLM推理的学者来说，当得出关于LLM潜在推理的结论时，必须小心避免既虚假的成功（源于记忆和推理捷径）又虚假的失败（可能源于人工实验设置，与前沿LLM的训练设置脱节）。
英文摘要：
Large language models can use chain-of-thought (CoT) to externalize reasoning, potentially enabling oversight of capable LLM agents. Prior work has shown that models struggle at two-hop question-answering without CoT. This capability is so basic that if it was a fundamental limitation, it would imply that many complex agentic tasks would similarly require CoT. We investigate LLM latent reasoning capabilities using two-hop question answering as a case study. Previous work on the gap between latent and externalized two-hop reasoning produced mixed evidence with inconclusive results. In this paper, we introduce a controlled setting for investigating two-hop reasoning in LLMs, where a positive result provides definitive evidence for latent reasoning. We fine-tune LLMs (including Llama 3 8B and GPT-4o) on synthetic facts and test two-hop reasoning over these facts. By using synthetic facts, we rule out memorization and reasoning shortcuts as explanations for two-hop performance. We observe a nuanced picture: Models fail to compose two synthetic facts, but can succeed when one fact is synthetic and the other is natural. These results demonstrate that LLMs are undeniably capable of latent two-hop reasoning, although it remains unclear how this ability scales with model size. Finally, we highlight a lesson for researchers studying LLM reasoning: when drawing conclusions about LLM latent reasoning, one must be careful to avoid both spurious successes (that stem from memorization and reasoning shortcuts) and spurious failures (that may stem from artificial experimental setups, divorced from training setups of frontier LLMs).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
