语言模型是否意识到未选择的道路？词元级不确定性与隐藏状态动态
Are language models aware of the road not taken? Token-level uncertainty and hidden state dynamics
AI总结：
本文通过隐藏激活控制与预测思维链推理中的词元级不确定性，发现模型在未确定最终答案时更易被干预，且隐藏状态能隐式表征可能的推理路径空间。
摘要：
当语言模型生成文本时，对单个词元的选择可能会将其引向截然不同的推理路径，这使得不确定性难以量化。在本工作中，我们考虑推理语言模型是否在生成过程中表征了它们可能采取的替代路径。为了检验这一假设，我们使用隐藏激活来控制并预测语言模型在思维链推理过程中的不确定性。在实验中，我们发现模型在不同词元处的不确定性程度，与通过控制其激活来引导模型的难易程度之间存在明显的相关性。这表明，当模型存在可用的替代路径时——换句话说，当它尚未对某个特定最终答案做出承诺时——激活干预最为有效。我们还发现，隐藏激活可以预测模型未来的结果分布，证明模型隐式地表征了可能路径的空间。
英文摘要：
When a language model generates text, the selection of individual tokens might lead it down very different reasoning paths, making uncertainty difficult to quantify. In this work, we consider whether reasoning language models represent the alternate paths that they could take during generation. To test this hypothesis, we use hidden activations to control and predict a language model's uncertainty during chain-of-thought reasoning. In our experiments, we find a clear correlation between how uncertain a model is at different tokens, and how easily the model can be steered by controlling its activations. This suggests that activation interventions are most effective when there are alternate paths available to the model -- in other words, when it has not yet committed to a particular final answer. We also find that hidden activations can predict a model's future outcome distribution, demonstrating that models implicitly represent the space of possible paths.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
