Teaching Thinking Models to Reason with Tools: A Full-Pipeline Recipe for Tool-Integrated Reasoning
Zhejiang University(浙江大学);Shanghai AI Laboratory(上海人工智能实验室);Peking University(北京大学);The Chinese University of Hong Kong(香港中文大学);Tsinghua University(清华大学);University of Science and Technology of China(中国科学技术大学);Shanghai Jiao Tong University(上海交通大学)
Tool-integrated reasoning (TIR) offers a direct way to extend thinking models beyond the limits of text-only reasoning. Paradoxically, we observe that tool-enabled evaluation can degrade reasoning performance even when the strong thinking models make almost no actual tool calls. In this paper, we investigate how to inject natural tool-use behavior into a strong thinking model without sacrificing its no-tool reasoning ability, and present a comprehensive TIR recipe. We highlight that (i) the effectiveness of TIR supervised fine-tuning (SFT) hinges on the learnability of teacher trajectories, which should prioritize problems inherently suited for tool-augmented solutions; (ii) controlling the proportion of tool-use trajectories could mitigate the catastrophic forgetting of text-only reasoning capacity; (iii) optimizing for pass@k and response length instead of training loss could maximize TIR SFT gains while preserving headroom for reinforcement learning (RL) exploration; (iv) a stable RL with verifiable rewards (RLVR) stage, built upon suitable SFT initialization and explicit safeguards against mode collapse, provides a simple yet remarkably effective solution. When applied to Qwen3 thinking models at 4B and 30B scales, our recipe yields models that achieve state-of-the-art performance in a wide range of benchmarks among open-source models, such as 96.7% and 99.2% on AIME 2025 for 4B and 30B, respectively.
S$^2$R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning
Tencent(腾讯);Tsinghua University(清华大学);The University of Hong Kong(香港大学);Fudan University(复旦大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
University of Science and Technology of China(中国科学技术大学);Alibaba Inc.(阿里巴巴公司);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen International Center for Industrial and Applied Mathematics(深圳国际工业与应用数学中心);Shenzhen Research Institute of Big Data(深圳大数据研究院)
The Markovian Thinker: Architecture-Agnostic Linear Scaling of Reasoning
Mila;Microsoft Research(微软研究院);McGill University(麦吉尔大学);ServiceNow Research(ServiceNow研究院);Canada CIFAR AI Chair(加拿大CIFAR人工智能主席);Chandar Research Lab(Chandar研究实验室);Polytechnique Montréal(蒙特利尔理工学院);Université de Montréal(蒙特利尔大学)
University of Science and Technology of China(中国科学技术大学);Alibaba Inc.(阿里巴巴);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen Research Institute of Big Data(深圳大数据研究院)