解锁潜在推理的黑箱：一种可解释性引导的干预方法
Unlocking the Black Box of Latent Reasoning: An Interpretability-Guided Approach to Intervention
AI总结：
本文通过结构、因果和几何探针分析潜在推理向量的可解释性，并基于此提出无需训练的解码时干预方法，提升大语言模型推理准确性。
摘要：
潜在推理使大型语言模型（LLMs）能够在连续隐藏状态内执行多步推理，相比显式思维链（CoT）提供了效率提升。然而，这些连续思维向量的不透明性阻碍了其可靠性和可控性。本文弥合了机械可解释性与可操作控制之间的差距。我们首先使用结构、因果和几何探针进行系统分析，揭示潜在向量编码了推理步骤的压缩、忠实表示，其中早期向量作为关键因果枢纽。在此基础上，我们将这些可解释性见解操作化为一套无需训练、解码时干预的方法，通过施加已识别的几何和语义先验来优化潜在推理过程。跨多个模型规模和不同任务领域的广泛实验表明，我们的方法持续提高了推理准确性。我们的可解释性引导干预一致地解锁了潜在能力，并在没有任何参数更新的情况下提高了推理准确性。
英文摘要：
Latent reasoning enables Large Language Models (LLMs) to perform multi-step inference within continuous hidden states, offering efficiency gains over explicit Chain-of-Thought (CoT). However, the opacity of these continuous thought vectors hinders their reliability and controllability. This paper bridges the gap between mechanistic interpretability and actionable control. We first present a systematic analysis using structural, causal, and geometric probes, revealing that latent vectors encode compressed, faithful representations of reasoning steps, with early vectors acting as critical causal hubs. Building on this, we operationalize these interpretability insights into a suite of training-free, decode-time interventions that refine the latent reasoning process by imposing the identified geometric and semantic priors. Extensive experiments across multiple model scales and diverse task domains demonstrate that our approaches consistently improve reasoning accuracy. Our interpretability-guided interventions consistently unlock latent capabilities and improve reasoning accuracy without any parameter updates.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
