arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.07237cs.CL
收藏

教语言模型用代码思考

Teaching Language Models to Think in Code

Korea University(韩国大学);AIGEN Sciences(AIGEN科技)

Hyeon Hwang, Jiwoo Lee, Jaewoo Kang

AI总结:

本文提出ThinC框架,通过代码自身进行推理而非依赖自然语言,提升了数学问题解决能力,其在多个基准测试中表现优异。

摘要:

工具集成推理(TIR)已成为语言模型中数学问题解决的主导范式,结合自然语言推理与代码执行。然而,这种交错设置存在三个关键限制:代码常作为事后验证器,中间自然语言计算易出错,且自然语言与代码角色重叠而非明确区分。我们提出ThinC(代码中的思考),一种框架中代码本身作为推理者而非由自然语言调用的工具。ThinC轨迹始于简短的自然语言规划步骤,之后所有推理通过仅通过执行输出连接的代码块展开。我们从教师模型中蒸馏了12.2k个代码导向的轨迹,并通过监督微调后接强化学习训练ThinC-1.7B和ThinC-4B。ThinC-4B在五个竞赛级数学基准测试中一致优于所有TIR基线,并甚至超越了更大的Qwen3-235B-A22B-Thinking。进一步分析显示,ThinC通过代码进行推理:其最终答案的99.2%基于解释器输出,且模型在代码执行失败时能可靠地恢复而不依赖中间自然语言推理。我们的代码和模型将很快发布。

英文摘要:

Tool-integrated reasoning (TIR) has emerged as a dominant paradigm for mathematical problem solving in language models, combining natural language (NL) reasoning with code execution. However, this interleaved setup has three key limitations: code often acts as a post-hoc verifier, intermediate NL computations are error-prone, and NL and code play overlapping rather than clearly distinct roles. We propose ThinC (Thinking in Code), a framework in which code itself serves as the reasoner rather than as a tool invoked by NL. A ThinC trajectory begins with a brief NL planning step, after which all reasoning unfolds through code blocks connected only by their execution outputs. We distill 12.2k code-centric trajectories from a teacher model and train ThinC-1.7B and ThinC-4B with supervised fine-tuning followed by reinforcement learning. ThinC-4B consistently outperforms every TIR baseline on five competition-level math benchmarks and even surpasses the much larger Qwen3-235B-A22B-Thinking. Further analysis shows that ThinC reasons through code: 99.2% of its final answers are grounded in interpreter output, and the model recovers reliably from code execution failures without intermediate NL reasoning. Our code and models will be released soon.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑