University of Science and Technology of China(中国科学技术大学);Alibaba Inc.(阿里巴巴公司);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen International Center for Industrial and Applied Mathematics(深圳国际工业与应用数学中心);Shenzhen Research Institute of Big Data(深圳大数据研究院)
Large reasoning models (LRMs) like OpenAI-o1 have shown impressive capabilities in natural language reasoning. However, these models frequently demonstrate inefficiencies or inaccuracies when tackling complex mathematical operations. While integrating computational tools such as Code Interpreters (CIs) offers a promising solution, it introduces a critical challenge: a conflict between the model's internal, probabilistic reasoning and the external, deterministic knowledge provided by the CI, which often leads models to unproductive deliberation. To overcome this, we introduce CoRT (Code-Optimized Reasoning Training), a post-training framework designed to teach LRMs to effectively utilize CIs. We propose \emph{Hint-Engineering}, a new data synthesis strategy that strategically injects diverse hints at optimal points within reasoning paths. This approach generates high-quality, code-integrated reasoning data specifically tailored to optimize LRM-CI interaction. Using this method, we have synthesized 30 high-quality samples to post-train models ranging from 1.5B to 32B parameters through supervised fine-tuning. CoRT further refines the multi-round interleaving of external CI usage and internal thinking by employing rejection sampling and reinforcement learning. Our experimental evaluations demonstrate CoRT's effectiveness, yielding absolute improvements of 4\% and 8\% on DeepSeek-R1-Distill-Qwen-32B and DeepSeek-R1-Distill-Qwen-1.5B, respectively, across five challenging mathematical reasoning datasets. Moreover, CoRT significantly enhances efficiency, reducing token usage by approximately 30\% for the 32B model and 50\% for the 1.5B model compared to pure natural language reasoning baselines. The models and code are available at: https://github.com/ChengpengLi1003/CoRT.
Teaching Algorithmic Reasoning via In-context Learning
Mila(米拉研究所);Université de Montreal(蒙特利尔大学);Google Research(谷歌研究院)
大语言模型(LLM)通过扩展模型和数据大小,显示出越来越多的上下文学习能力。尽管取得了这些进展,LLM仍然无法解决算法推理问题。虽然为最终答案提供了一个基本原理,导致了多步推理问题的进一步改进,但Anil et al. 2022表明,即使是简单的算法推理任务,如奇偶校验,也远未解决。在这项工作中,我们确定并研究了成功教授算法推理的四个关键阶段:(1)将算法制定为技能,(2)同时教授多种技能(技能积累),(3)教授如何组合技能(技能组合)和(4)教授如何使用技能作为工具。我们表明,它是可以教算法推理LLMs通过上下文学习,我们称之为算法提示。我们评估我们的方法在各种算术和定量推理任务,并表现出显着的提升性能比现有的提示技术。特别是,对于长奇偶校验、加法、乘法和减法,与最佳可用基线相比,我们分别实现了约10倍、9倍、5倍和2倍的误差减少。
Bayesian Teaching Enables Probabilistic Reasoning in Large Language Models
MIT(麻省理工学院);Meta;Google(谷歌公司);DeepMind University of British Columbia(不列颠哥伦比亚大学深度思维学院);Vector Institute(向量研究所);New York University(纽约大学);Google Research(谷歌研究)
Teaching LLMs According to Their Aptitude: Adaptive Reasoning for Mathematical Problem Solving
The Hong Kong University of Science and Technology(香港科学与技术大学);Huawei Noah’s Ark Lab(华为诺亚实验室);Zhejiang University(浙江大学);Peking University(北京大学);The Hong Kong Polytechnic University(香港理工大学);University of Surrey(萨里大学)
Teaching LLM to Reason: Reinforcement Learning from Algorithmic Problems without Code
University of Science and Technology of China(中国科学技术大学);Alibaba Group(阿里巴巴集团);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Tailored Teaching with Balanced Difficulty: Elevating Reasoning in Multimodal Chain-of-Thought via Prompt Curriculum
Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Hunan Vanguard Group Corporation Co., Ltd.(湖南 Vanguard集团有限公司);Huaneng Information Technology Co., Ltd.(华能信息技术有限公司);Tsinghua University(清华大学);Zhejiang University(浙江大学);PowerChina Huadong Engineering Co., Ltd.(中国电建华东工程有限公司)
Teaching Audio Models to Reason: A Unified Framework for Source- and Layer-wise Distillation
Jiutian Artificial Intelligence Research Institute, China Mobile, Beijing, China(九天人工智能研究院,中国移动,北京);The State Key Laboratory of Multimedia Information Processing, Peking University, Beijing, China(多媒体信息处理国家重点实验室,北京大学,北京)
Teaching Thinking Models to Reason with Tools: A Full-Pipeline Recipe for Tool-Integrated Reasoning
Zhejiang University(浙江大学);Shanghai AI Laboratory(上海人工智能实验室);Peking University(北京大学);The Chinese University of Hong Kong(香港中文大学);Tsinghua University(清华大学);University of Science and Technology of China(中国科学技术大学);Shanghai Jiao Tong University(上海交通大学)
UKP-SQuARE: An Interactive Tool for Teaching Question Answering
Ubiquitous Knowledge Processing Lab (UKP Lab)(普适知识处理实验室(UKP实验室));Department of Computer Science(计算机科学系);Hessian Center for AI (hessian.AI)(黑森人工智能中心(hessian.AI));Technical University of Darmstadt(达姆施塔特工业大学)
Teaching Code LLMs to Reason with Intermediate Formal Specifications
FPT Software AI Center(FPT软件人工智能中心);University of Texas at Dallas(德克萨斯大学达拉斯分校);Hanoi Univ. of Science and Tech.(河内科学技术大学);Texas, USA(得克萨斯州,美国);Hanoi, Vietnam(河内,越南)