arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.07159cs.AI
收藏

通过不确定性最小化提升推理时间的推理能力

Improving reasoning at inference time via uncertainty minimisation

Aarhus University(奥胡斯大学)

Nicolas Legrand, Kenneth Enevoldsen, Márton Kardos, Kristoffer Nielbo

AI总结:

通过在单个思想层面最大化自我确定性,提升推理能力并实现高效推理时间扩展。

摘要:

大型语言模型(LLMs)现在表现出强大的多步推理能力,但现有的推理时间扩展方法仍然计算成本高,通常依赖于广泛的采样或外部评估者。我们提出了一种系统性的策略,将推理视为不确定性最小化,并在单个思想的层面进行操作,而不是在token层面。我们的方法在每个推理步骤中选择使模型自我确定性最大的延续,该指标由其内部预测分布计算得出。这种方法在少量样本下实现了显著提升,仅依赖模型内部信号,并适用于开放性问题,而不是多数投票等方法。在MATH500和GSM8K上多个模型大小的实验表明,思想层面的自我确定性最大化在很大程度上优于贪婪解码,并在可比的token预算下与自我一致性相当或更优。跨语言评估进一步表明,该方法在高资源语言之外也具有稳健的迁移能力。此外,对自我确定性动态的分析表明,正确的推理轨迹早期会收敛到稳定路径,这表明早期决策,可能与推理过程的规划有关,是最终准确性预测的因素。基于这一结果,我们展示了在早期步骤应用自我确定性最大化可以解释大部分性能提升,并提供了一种简单而高效的推理时间扩展方法。

英文摘要:

Large language models (LLMs) now exhibit strong multi-step reasoning abilities, but existing inference-time scaling methods remain computationally expensive, often relying on extensive sampling or external evaluators. We propose a principled strategy that frames reasoning as uncertainty minimisation and operates at the level of individual thoughts rather than tokens. Our method selects, at each reasoning step, the continuation that maximizes the model's self-certainty, a metric computed from its internal predictive distribution. This approach achieves significant improvement with a small number of samples, relies exclusively on model-internal signals, and applies to open-ended questions as opposed to methods like majority voting. Experiments on MATH500 and GSM8K across multiple model sizes demonstrate that thought-level self-certainty maximization consistently outperforms greedy decoding and matches or exceeds self-consistency under comparable token budgets. Cross-linguistic evaluations further indicate that the method transfers robustly beyond high-resource languages. Furthermore, analysis of self-certainty dynamics reveals that correct reasoning trajectories converge early to stable paths, suggesting that early decisions, likely associated with the planning of the reasoning process, are predictive of final accuracy. Building on this result, we show that self-certainty maximisation applied to the early steps can explain most of the performance gain and provide a simple yet efficient inference-time scaling method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑