ThinkEdit：通过可解释权重编辑缓解推理模型中过度简短的思考
ThinkEdit: Interpretable Weight Editing to Mitigate Overly Short Thinking in Reasoning Models
AI总结：
本文发现LLM推理长度受隐藏表示中的线性方向控制，并提出ThinkEdit，通过编辑约4%注意力头、仅改动0.2%参数来移除短推理方向，显著提升数学准确率。
摘要：
近期研究表明，借助思维链（CoT）推理增强的大型语言模型（LLM）展现出令人瞩目的问题解决能力。然而，在本工作中，我们发现一个反复出现的问题：这些模型偶尔会生成过度简短的推理，导致即使在简单数学问题上性能也会下降。具体而言，我们研究了推理长度如何嵌入推理模型的隐藏表示中，及其对准确率的影响。分析表明，推理长度由表示空间中的一个线性方向控制，这使我们能够通过沿该方向引导模型来诱发过度简短的推理。基于这一洞见，我们提出 ThinkEdit，这是一种简单而有效的权重编辑方法，用于缓解过度简短推理的问题。我们首先识别出一小部分注意力头（约4%），它们主要驱动简短推理行为。随后，我们编辑这些注意力头的输出投影权重，以移除简短推理方向。仅更改模型0.2%的参数，ThinkEdit 便有效减少了过度简短推理，并使简短推理输出的准确率显著提升（+6.39%），同时在多个数学基准上带来整体改进（+3.34%）。我们的发现为 LLM 内部如何控制推理长度提供了新的机制性洞见，并凸显了细粒度模型干预在提升推理质量方面的潜力。我们的代码可在 https://github.com/Trustworthy-ML-Lab/ThinkEdit 获取。
英文摘要：
Recent studies have shown that Large Language Models (LLMs) augmented with chain-of-thought (CoT) reasoning demonstrate impressive problem-solving abilities. However, in this work, we identify a recurring issue where these models occasionally generate overly short reasoning, leading to degraded performance on even simple mathematical problems. Specifically, we investigate how reasoning length is embedded in the hidden representations of reasoning models and its impact on accuracy. Our analysis reveals that reasoning length is governed by a linear direction in the representation space, allowing us to induce overly short reasoning by steering the model along this direction. Building on this insight, we introduce ThinkEdit, a simple yet effective weight-editing approach to mitigate the issue of overly short reasoning. We first identify a small subset of attention heads (approximately 4%) that predominantly drive short reasoning behavior. We then edit the output projection weights of these heads to remove the short reasoning direction. With changes to only 0.2% of the model's parameters, ThinkEdit effectively reduces overly short reasoning and yields notable accuracy gains for short reasoning outputs (+6.39%), along with an overall improvement across multiple math benchmarks (+3.34%). Our findings provide new mechanistic insights into how reasoning length is controlled within LLMs and highlight the potential of fine-grained model interventions to improve reasoning quality. Our code is available at: https://github.com/Trustworthy-ML-Lab/ThinkEdit
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
