arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.15412cs.CL
收藏

从候选 token 到最终结果:从推理模型表征中解码相对长程候选值

From Candidate Tokens to Final Outcomes: Decoding Relative Long-Horizon Candidate Value from Reasoning Model Representations

WeChat AI, Tencent Inc.(腾讯微信人工智能)

Yixin Yang, Yue Wang, Fandong Meng, Zhifang Sui

AI总结:

该研究构建RCTV数据集,通过线性探针证明推理模型的内部表征可线性解码相对长程候选值,且用解码信号指导token选择能提升推理成功概率。

摘要:

大型推理模型会逐 token 生成推理轨迹。在给定前缀时,多个下一个 token 候选可能看似合理,但会影响后续推理过程,且导致正确答案的概率各不相同。这些差异是否体现在内部表征中仍不明确。为研究该问题,我们定义相对长程候选值以量化同一前缀内的差异,并构建了推理候选 token 值(Reasoning Candidate-Token Value, RCTV)数据集,将数学推理前缀与合理的下一个 token 候选及其经验长程候选值配对。利用 RCTV,我们通过稀疏线性探针检查候选位置的隐藏状态、多层感知机输出和注意力输出。实验结果提供了证据,表明相对长程候选值可从多个层的这些表征中线性解码。候选值探针的性能也优于基于概率和嵌入的基线。此外,一项受控在线干预实验显示,在生成过程中使用解码信号指导 token 选择可提高最终成功的概率。我们的发现提供了证据,表明内部表征编码了相对长程候选值的信息,为局部候选级表征如何与长程推理结果关联提供了见解。

英文摘要:

Large reasoning models generate reasoning trajectories token by token. At a given prefix, multiple next-token candidates may be plausible, yet shape subsequent reasoning and differ in their chances of leading to the correct answer. Whether these differences are reflected in internal representations remains unclear. To study this question, we define relative long-horizon candidate value to quantify within-prefix differences and construct the Reasoning Candidate-Token Value (RCTV) dataset, pairing mathematical reasoning prefixes with plausible next-token candidates and their empirical long-horizon candidate values. Using RCTV, we examine hidden states, multilayer perceptron outputs, and attention outputs at candidate positions with sparse linear probes. Experimental results provide evidence that relative long-horizon candidate value is linearly decodable from these representations at multiple layers. Candidate-value probes can also outperform probability- and embedding-based baselines. Furthermore, a controlled online intervention experiment shows that using the decoded signals to guide token selection during generation can improve the chances of eventual success. Our findings provide evidence that internal representations encode information about relative long-horizon candidate value, offering insights into how local candidate-level representations relate to long-horizon reasoning outcomes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑