超越链式推理：大型语言模型中的一个潜在计算模式
Triggering Chain-of-Thought via Latent Feature Interventions in Large Language Models
AI总结：
研究发现大型语言模型中的推理能力由潜在内部激活支持，通过引导特定潜在特征可提升性能，CoT提示并非唯一触发方式。
摘要：
链式推理（CoT）提示已经提升了大型语言模型（LLMs）的推理性能，但仍然不清楚为什么它有效，以及它是否是触发大型语言模型推理的唯一机制。在本工作中，我们通过直接分析和干预LLMs的内部表示，使用稀疏自编码器（SAEs）来研究这个问题，识别出一组与LLM推理行为因果相关的潜在特征。在多个模型家族和推理基准中，我们发现引导一个与推理相关的潜在特征可以显著提高准确性，而无需显式的CoT提示。对于大型模型，潜在引导在性能上与标准CoT提示相当，但产生更高效的输出。我们进一步观察到，这种以推理为导向的内部状态在生成早期被触发，并且可以覆盖抑制显式推理的提示级指令。总体而言，我们的结果表明，LLMs的多步推理是由可以被外部激活的潜在内部激活所支持的，而CoT提示是激活这一机制的一种有效但不唯一的方式，而不是其必要原因。
英文摘要：
Chain-of-Thought (CoT) prompting often improves the reasoning performance of large language models (LLMs), but the internal signal that triggers this behavior remains poorly understood. Leveraging the sparse features captured by Sparse Autoencoders (SAEs), we propose a systematic framework to analyze and intervene on the internal representations of LLMs, identifying a small set of latent features that are linked to reasoning behavior and can be causally tested through targeted intervention. Across multiple model families and reasoning benchmarks, we show that steering one or a small number of reasoning-related latent features can substantially induce reasoning behavior without explicit CoT prompting, achieving accuracy comparable to CoT. We further show that the identified features are not tied to particular wording patterns or verbosity, and confirm their causal role in reasoning through suppression experiments that impair performance even under CoT prompting. These results suggest that CoT prompting activates specific latent features to trigger reasoning, and that targeted intervention on these features offers an alternative pathway to elicit efficient reasoning behavior without explicit CoT prompting. Code is available at https://github.com/Zhenghao-He/LatentCoT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
