arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.14528cs.AIcs.CL
收藏

解析大语言模型推理中的故障动态

Dissecting Failure Dynamics in Large Language Model Reasoning

School of Information Science and Engineering, Yunnan University, Kunming, China(云南大学信息科学与工程学院,昆明,中国);Yunnan Key Laboratory of Intelligent Systems and Computing, Kunming, China(云南省智能系统与计算重点实验室,昆明,中国)

Wei Zhu, Jian Zhang, Lixing Yu, Kun Yue, Zhiwen Tang

AI总结:

研究揭示大语言模型推理错误多源于早期少量转换点,提出GUARD框架通过不确定性信号引导干预,提升推理可靠性。

摘要:

大型语言模型(LLMs)通过延长推理时间进行深入思考而表现出色,但其推理错误的产生机制仍不明确。通过分析模型生成的推理轨迹,发现错误并非均匀分布,而是往往起源于少量早期转换点,之后推理在局部上保持一致但整体错误。这些转换点与token层面的熵骤增吻合,从相同中间状态的不同延续路径仍可能得到正确结果。基于这些观察,我们引入GUARD,一种针对推理时间的框架,利用不确定性信号探测并引导关键转换。在多个基准测试中的实证评估证实,由这些故障动态引导的干预可产生更可靠的推理结果。我们的发现强调了理解推理何时及如何首次偏离的重要性,补充了现有仅关注扩展推理时间计算的方法。

英文摘要:

Large Language Models (LLMs) achieve strong performance through extended inference-time deliberation, yet how their reasoning failures arise remains poorly understood. By analyzing model-generated reasoning trajectories, we find that errors are not uniformly distributed but often originate from a small number of early transition points, after which reasoning remains locally coherent but globally incorrect. These transitions coincide with localized spikes in token-level entropy, and alternative continuations from the same intermediate state can still lead to correct solutions. Based on these observations, we introduce GUARD, a targeted inference-time framework that probes and redirects critical transitions using uncertainty signals. Empirical evaluations across multiple benchmarks confirm that interventions guided by these failure dynamics lead to more reliable reasoning outcomes. Our findings highlight the importance of understanding when and how reasoning first deviates, complementing existing approaches that focus on scaling inference-time computation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑