arXiv 2601.142702026-01-02cs.CLcs.AI

揭开黑箱：关于大语言模型多步推理机制的综述

Opening the Black Box: A Survey on the Mechanisms of Multi-Step Reasoning in Large Language Models

Peking University（北京大学）；Stanford University（斯坦福大学）；Tsinghua University（清华大学）；University of Arizona（亚利桑那大学）；National University of Singapore（新加坡国立大学）

Liangming Pan, Jason Liang, Jiaran Ye, Minglai Yang, Xinyuan Lu, Fengbin Zhu

备注：: Technical Report

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文综述了大语言模型多步推理机制，探讨了隐式和显式推理过程，并提出了未来研究方向。

摘要：

大语言模型（LLMs）已展现出解决需要多步推理问题的显著能力，但使这种能力得以实现的内部机制仍不明确。不同于现有综述主要聚焦于提升性能的工程方法，本文提供了对LLM多步推理机制的全面概述。我们围绕由七个相互关联的研究问题构成的概念框架组织综述，从LLMs如何在隐藏激活中执行隐式多跳推理到如何通过口头化的显式推理重塑内部计算。最后，我们强调了五个未来机制研究的研究方向。

英文摘要：

Large Language Models (LLMs) have demonstrated remarkable abilities to solve problems requiring multiple reasoning steps, yet the internal mechanisms enabling such capabilities remain elusive. Unlike existing surveys that primarily focus on engineering methods to enhance performance, this survey provides a comprehensive overview of the mechanisms underlying LLM multi-step reasoning. We organize the survey around a conceptual framework comprising seven interconnected research questions, from how LLMs execute implicit multi-hop reasoning within hidden activations to how verbalized explicit reasoning remodels the internal computation. Finally, we highlight five research directions for future mechanistic studies.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。