逻辑运算激发长推理能力而不需训练
Logit Arithmetic Elicits Long Reasoning Capabilities Without Training
AI总结：
本文提出ThinkLogit方法，通过逻辑运算将小型推理引导器的能力转移至大模型，无需训练即可提升推理性能，实验显示在六个基准测试中实现21.5%-24.2%的提升。
摘要：
大型推理模型通过复杂的策略如回溯和自我验证展现出长链推理能力，但通常需要资源密集的后期训练。本文研究是否能在无需梯度更新的情况下在大型模型中激发此类行为。为此，我们提出在解码时的ThinkLogit方法，利用逻辑运算将这些能力从显著较小的推理引导器转移到大型非推理目标。我们进一步表明，通过训练引导器纠正目标的错误，使用偏好优化混合模型输出，可以提升性能，这种设置称为ThinkLogit-DPO。我们使用Qwen2.5-32B模型，由R1-Distill-Qwen-1.5B引导，在数学、科学和编码领域六个推理基准上评估这些方法。实验表明，ThinkLogit和ThinkLogit-DPO分别在目标模型上实现21.5%和24.2%的相对提升。此外，ThinkLogit即使在引导器和目标来自不同模型家族时仍有效。关键的是，我们的方法要求大型模型无需训练，并且在计算logits时并行处理时会带来最小的推理开销，为大规模长推理提供了实用解决方案。
英文摘要：
Large reasoning models exhibit long chain-of-thought reasoning with complex strategies such as backtracking and self-verification. Yet, these capabilities typically require resource-intensive post-training. We investigate whether such behaviors can be elicited in large models without any gradient updates. To this end, we propose a decoding-time approach, ThinkLogit, which utilizes logit arithmetic to transfer these capabilities from a substantially smaller reasoning guider to a large non-reasoning target. We further show that we can boost performance by training the guider to correct the target's errors using preference optimization over mixed model outputs, a setup we refer to as ThinkLogit-DPO. We evaluate these methods across six reasoning benchmarks spanning math, science, and coding domains using the Qwen2.5-32B guided by R1-Distill-Qwen-1.5B, a model 21x smaller. Our experiments demonstrate that ThinkLogit and ThinkLogit-DPO achieve a relative improvement of 21.5% and 24.2%, respectively, over the target model. Moreover, ThinkLogit remains effective even when the guider and target come from different model families. Crucially, our method requires zero training for the large model and would incur minimal inference overhead when logits are computed in parallel, presenting a practical solution for enabling long reasoning at scale.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
