arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2511.00536cs.CL
收藏

Word Salad Chopper:推理模型明知故犯地在无用的重复上浪费大量解码预算

Word Salad Chopper: Reasoning Models Waste A Ton Of Decoding Budget On Useless Repetitions, Self-Knowingly

University of Minnesota(明尼苏达大学);Rice University(里士满大学);Stevens Institute of Technology(史蒂文斯理工学院);Lambda, Inc(Lambda公司)

Wenya Xie, Shaochen, Zhong, Hoang Anh Duy Le, Zhaozhuo Xu, Jianwen Xie, Zirui Liu

AI总结:

本文发现大型推理模型(LRM)会自我意识到地陷入无用的“word salad”重复循环,并提出轻量级组件 WordSaladChopper,通过单层线性分类器检测并截断冗余 token,在最小质量损失下大幅节省解码预算。

摘要:

大型推理模型(LRM)通常受限于输出 token 的高昂成本。我们证明,这些 token 中有很大一部分是无用的自我重复——我们称之为“word salad”——它们耗尽了解码预算却没有增加任何价值。有趣的是,我们观察到 LRMs 在陷入这些循环时具有自我意识:每个推理块末尾的 <\ \ > token 的隐藏状态表现出特定的模式,使我们能够通过单层线性分类器实时检测 word salad 行为。一旦检测到,只需进行简单的截断并附加一个直接的重新生成提示,就能大幅节省长度,同时质量损失极小。我们的工作提出了 WordSaladChopper(WSC)——一个轻量级、即插即用的 LRM 组件,它仅通过移除语义冗余的 token,对推理轨迹的侵入性极小。鉴于其低开销、强节省效果以及 word salad token 缺乏语义价值,我们认为,对于所有注重用户体验的 LRM 应用而言,WSC 或类似组件是必不可少的。我们的代码已在 https://github.com/wenyaxie023/WordSaladChopper 公开。

英文摘要:

Large Reasoning Models (LRMs) are often bottlenecked by the high cost of output tokens. We show that a significant portion of these tokens are useless self-repetitions - what we call "word salad" - that exhaust the decoding budget without adding value. Interestingly, we observe that LRMs are self-aware when trapped in these loops: the hidden states of <\n\n> tokens trailing each reasoning chunk exhibit patterns that allow us to detect word salad behavior on-the-fly via a single-layer linear classifier. Once detected, a simple chop appended by a straightforward regeneration prompt yields substantial length savings with minimal quality loss. Our work offers WordSaladChopper (WSC) - a lightweight, turnkey component for LRM that is minimally invasive to its reasoning trajectory by only removing semantically redundant tokens. Given its low overhead, strong savings, and the lack of semantic value of word salad tokens, we believe it is not too far-fetched to argue that WSC - or a similar component - is a must-have for all LRM applications with user experience in mind. Our code is publicly available at https://github.com/wenyaxie023/WordSaladChopper.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑