arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.23926cs.AIcs.LG
收藏

多少思考才足够?量化和理解LLM推理中的冗余

How Much Thinking is Enough? Quantifying and Understanding Redundancy in LLM Reasoning

Fudan University(复旦大学);The Chinese University of Hong Kong(香港中文大学)

Zhiyuan Zhai, Xinkai You, Wenjing Yan, Xin Wang

AI总结:

本文通过形式化推理冗余度量,量化了前沿推理模型在数学基准上高达61%-93%的步骤级冗余,并证明这种冗余是长度无关结果奖励的结构性后果,而非模型特定伪影。

摘要:

具备推理能力的大语言模型通过生成长思维链来解决难题,这严重增加了延迟、GPU时间和能耗。粗略检查其轨迹发现大量重构、验证和循环自省,然而这种深思熟虑中有多少实际上是必要的,从未在大规模上被度量或从第一性原理解释。本文填补了这两个空白。 我们直接以推理模型本身的形式化推理冗余:一个正确轨迹的冗余度是其尾部可被截断的最大分段步骤比例,同时迫使模型终止思考并输出最终答案,仍能产生正确答案。对四个前沿推理模型和两个数学基准的大规模量化表明,步骤级冗余一致地高——在我们研究的8个(模型,基准)条件下介于61%和93%之间,其中六个条件下中位关键前缀等于单个分段步骤——该发现对评判模型族的选择是稳健的,并且尽管在MATH-500上随问题难度增加而降低,所有四个模型即使在最难的Level-5问题上仍然显著冗余(ρ∈[46%,85%])。 然后我们证明这种冗余是长度无关结果奖励的结构性后果,而非模型特定伪影:在任何此类奖励下,没有有限期望停止时间是最优的。该结果无论RL算法、基础模型、数据分布或策略是通过RL还是蒸馏获得均成立;因此过度思考不是需要在单个模型中修补的缺陷,而是当前推理模型训练方式的结构性属性。代码:https://github.com/zhiyuanZhai20/how-much-thinking-is-enough

英文摘要:

Reasoning-capable large language models solve hard problems by emitting long chains of thought, paying heavily in latency, GPU time, and energy. Casual inspection of their traces reveals extensive reformulation, verification, and circular self-reflection, yet how much of this deliberation is actually necessary has never been measured at scale or explained from first principles. This paper closes both gaps. We formalise reasoning redundancy directly in terms of the reasoning model itself: the redundancy of a correct trace is the largest fraction of its trailing segmented steps that can be truncated while $π$, forced to terminate thinking and emit a final answer, still produces the correct answer. A large-scale quantification across four frontier reasoning models and two mathematical benchmarks shows that step-level redundancy is consistently high -- between 61% and 93% across the 8 (model, benchmark) conditions we study, with the median critical prefix equal to a single segmented step in six of the eight conditions -- that the finding is robust to the choice of judge family, and that although $ρ$ decreases with problem difficulty on MATH-500, all four models remain substantially redundant ($ρ\in [46\%, 85\%]$) even on the hardest Level-5 problems. We then prove that this redundancy is a structural consequence of length-agnostic outcome rewards, not a model-specific artefact: under any such reward, no finite expected stopping time is optimal. The result holds regardless of RL algorithm, base model, data distribution, or whether the policy is obtained via RL or distillation; over-thinking is therefore not a bug to be patched in individual models but a structural property of how current reasoning models are trained. Code: https://github.com/zhiyuanZhai20/how-much-thinking-is-enough

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑