arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.04902cs.LG
收藏

潜在推理模型是否易于解释?

Are Latent Reasoning Models Easily Interpretable?

Connor Dilgren, Sarah Wiegreffe

AI总结:

本文研究了潜在推理模型的可解释性,发现其推理令牌往往不必要,且可通过解码自然语言推理轨迹验证预测正确性,表明当前LRMs主要编码可解释过程。

摘要:

潜在推理模型(LRMs)因其低推理成本和理论能力探索多条推理路径并行而受到广泛关注。然而,这些优势以可解释性降低为代价:LRMs难以监控,因为它们不以自然语言推理。本文通过研究两种最先进的LRMs,发现潜在推理令牌往往对预测不必要;在逻辑推理数据集上,LRMs几乎总能产生相同最终答案而无需使用潜在推理。这种推理令牌的低利用率可能部分解释了为什么LRMs不一致地优于显式推理方法,并对先前工作中这些令牌的作用提出质疑。其次,当潜在推理令牌对性能必要时,我们可以在65-93%的时间解码正确预测实例的黄金推理轨迹。这表明LRMs通常实现预期解决方案而非不可解释的推理过程。最后,我们提出了一种方法,无需先验知识黄金推理轨迹即可从潜在令牌中解码验证的自然语言推理轨迹,证明对于大多数正确预测,可以找到验证轨迹,但对于错误预测仅少数情况。我们的发现表明,当前LRMs大多编码可解释过程,而可解释性本身可以作为预测正确性的信号。

英文摘要:

Latent reasoning models (LRMs) have attracted significant research interest due to their low inference cost (relative to explicit reasoning models) and theoretical ability to explore multiple reasoning paths in parallel. However, these benefits come at the cost of reduced interpretability: LRMs are difficult to monitor because they do not reason in natural language. This paper presents an investigation into LRM interpretability by examining two state-of-the-art LRMs. First, we find that latent reasoning tokens are often unnecessary for LRMs' predictions; on logical reasoning datasets, LRMs can almost always produce the same final answers without using latent reasoning at all. This underutilization of reasoning tokens may partially explain why LRMs do not consistently outperform explicit reasoning methods and raises doubts about the stated role of these tokens in prior work. Second, we demonstrate that when latent reasoning tokens are necessary for performance, we can decode gold reasoning traces up to 65-93% of the time for correctly predicted instances. This suggests LRMs often implement the expected solution rather than an uninterpretable reasoning process. Finally, we present a method to decode a verified natural language reasoning trace from latent tokens without knowing a gold reasoning trace a priori, demonstrating that it is possible to find a verified trace for a majority of correct predictions but only a minority of incorrect predictions. Our findings highlight that current LRMs largely encode interpretable processes, and interpretability itself can be a signal of prediction correctness.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑