内在思维链：大语言模型中分层子任务调度的实证证据
Internal Chain-of-Thought: Empirical Evidence for Layer-wise Subtask Scheduling in LLMs
AI总结：
本文通过层掩码、跨任务修补与LogitLens实验，证明LLM会在不同层学习并顺序执行子任务，为内在思维链和指令级激活引导提供证据。
摘要：
我们表明，大语言模型（LLM）表现出一种“内在思维链”：它们逐层顺序分解并执行复合任务。本研究基于两项主张：（i）不同子任务在不同网络深度被学习；（ii）这些子任务跨层顺序执行。在包含15个两步复合任务的基准上，我们采用层起始上下文掩码，并提出一种新颖的跨任务修补方法，证实了主张（i）。为检验主张（ii），我们应用LogitLens解码隐藏状态，揭示出一致的逐层执行模式。我们进一步在真实世界TRACE基准上重复分析，观察到相同的分步动态。总之，我们的结果通过展示LLM在内部规划和执行子任务（或指令）的能力，增强了LLM的透明度，并为细粒度、指令级的激活引导开辟了途径。
英文摘要：
We show that large language models (LLMs) exhibit an $\textit{internal chain-of-thought}$: they sequentially decompose and execute composite tasks layer-by-layer. Two claims ground our study: (i) distinct subtasks are learned at different network depths, and (ii) these subtasks are executed sequentially across layers. On a benchmark of 15 two-step composite tasks, we employ layer-from context-masking and propose a novel cross-task patching method, confirming (i). To examine claim (ii), we apply LogitLens to decode hidden states, revealing a consistent layerwise execution pattern. We further replicate our analysis on the real-world $\text{TRACE}$ benchmark, observing the same stepwise dynamics. Together, our results enhance LLMs transparency by showing their capacity to internally plan and execute subtasks (or instructions), opening avenues for fine-grained, instruction-level activation steering.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
