Can Large language models (LLMs) learn to reason without any weight update and only through in-context learning (ICL)? ICL is strikingly sample-efficient, often learning from only a handful of demonstrations, but complex reasoning tasks typically demand many training examples to learn from. However, naively scaling ICL by adding more demonstrations breaks down at this scale: attention costs grow quadratically, performance saturates or degrades with longer contexts, and the approach remains a shallow form of learning. Due to these limitations, practitioners predominantly rely on in-weight learning (IWL) to induce reasoning. In this work, we show that by using Prefix Tuning, LLMs can learn to reason without overloading the context window and without any weight updates. We introduce $\textbf{ReasonCACHE}$, an instantiation of this mechanism that distills demonstrations into a fixed key-value cache. Empirically, across challenging reasoning benchmarks, including GPQA-Diamond, ReasonCACHE outperforms standard ICL and matches or surpasses IWL approaches. Further, it achieves this all while being more efficient across three key axes: data, inference cost, and trainable parameters. We also theoretically prove that ReasonCACHE can be strictly more expressive than low-rank weight update since the latter ties expressivity to input rank, whereas ReasonCACHE bypasses this constraint by directly injecting key-values into the attention mechanism. Together, our findings identify ReasonCACHE as a middle path between in-context and in-weight learning, providing a scalable algorithm for learning reasoning skills beyond the context window without modifying parameters. Our project page: https://reasoncache.github.io/
Teaching Algorithmic Reasoning via In-context Learning
Mila(米拉研究所);Université de Montreal(蒙特利尔大学);Google Research(谷歌研究院)
大语言模型(LLM)通过扩展模型和数据大小,显示出越来越多的上下文学习能力。尽管取得了这些进展,LLM仍然无法解决算法推理问题。虽然为最终答案提供了一个基本原理,导致了多步推理问题的进一步改进,但Anil et al. 2022表明,即使是简单的算法推理任务,如奇偶校验,也远未解决。在这项工作中,我们确定并研究了成功教授算法推理的四个关键阶段:(1)将算法制定为技能,(2)同时教授多种技能(技能积累),(3)教授如何组合技能(技能组合)和(4)教授如何使用技能作为工具。我们表明,它是可以教算法推理LLMs通过上下文学习,我们称之为算法提示。我们评估我们的方法在各种算术和定量推理任务,并表现出显着的提升性能比现有的提示技术。特别是,对于长奇偶校验、加法、乘法和减法,与最佳可用基线相比,我们分别实现了约10倍、9倍、5倍和2倍的误差减少。
Bayesian Teaching Enables Probabilistic Reasoning in Large Language Models
MIT(麻省理工学院);Meta;Google(谷歌公司);DeepMind University of British Columbia(不列颠哥伦比亚大学深度思维学院);Vector Institute(向量研究所);New York University(纽约大学);Google Research(谷歌研究)
Teaching LLMs According to Their Aptitude: Adaptive Reasoning for Mathematical Problem Solving
The Hong Kong University of Science and Technology(香港科学与技术大学);Huawei Noah’s Ark Lab(华为诺亚实验室);Zhejiang University(浙江大学);Peking University(北京大学);The Hong Kong Polytechnic University(香港理工大学);University of Surrey(萨里大学)
Teaching AI Stepwise Diagnostic Reasoning with Report-Guided Chain-of-Thought Learning
Fujian University of Technology(福建工程学院);Fujian Provincial Key Laboratory of Big Data Mining and Applications(福建省大数据挖掘与应用重点实验室);Key Laboratory of Biomedical Imaging Science and System, Chinese Academy of Sciences(生物医学成像科学与系统重点实验室,中国科学院)
SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought
SIGS, Tsinghua University(清华大学信息科学与技术学院);Southern University of Science and Technology(南方科技大学);Guangdong Laboratory of AI and Digital Economy (SZ)(广东人工智能与数字经济实验室);The Hong Kong University of Science and Technology(香港科技大学);Peking University(北京大学)
The Challenge of Teaching Reasoning to LLMs Without RL or Distillation
Nvidia Corporation(Nvidia公司);Institute for Artificial Intelligence Research(人工智能研究机构);Faculty of Technical Sciences, University of Novi Sad(诺维萨德大学技术科学学院);Department of Mathematics and Informatics, Faculty of Sciences, University of Novi Sad(诺维萨德大学科学学院数学与信息学系);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Chicago(芝加哥大学);Georgia Institute of Technology(佐治亚理工学院);Department of Computer and Information Science, School of Engineering and Applied Science, University of Pennsylvania(宾夕法尼亚大学计算机与信息科学系);Massachusetts Institute of Technology(麻省理工学院)
Teaching the Way, Not the Answer: Privileged Tutoring Distillation for Multimodal Policy Optimization
Tianjin University(天津大学);Beijing Institute of Technology(北京理工大学);Singapore Management University(新加坡国立大学);University of Chinese Academy of Sciences(中国科学院大学);Xiaomi Inc(小米公司)
Tailored Teaching with Balanced Difficulty: Elevating Reasoning in Multimodal Chain-of-Thought via Prompt Curriculum
Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Hunan Vanguard Group Corporation Co., Ltd.(湖南 Vanguard集团有限公司);Huaneng Information Technology Co., Ltd.(华能信息技术有限公司);Tsinghua University(清华大学);Zhejiang University(浙江大学);PowerChina Huadong Engineering Co., Ltd.(中国电建华东工程有限公司)
Acting Less is Reasoning More! Teaching Model to Act Efficiently
The Chinese University of Hong Kong(香港中文大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴娜-香槟分校);Princeton University(普林斯顿大学);Sun Yat-sen University(中山大学);Hong Kong University of Science and Technology(香港科学理工大学)
Teaching large language models to reason like expert diagnosticians
Department of Biomedical Informatics, Harvard Medical School(哈佛医学院生物医学信息学系);Department of Medicine, Beth Israel Deaconess Medical Center(贝塞斯达医院内科部);The Mongan Institute, Massachusetts General Hospital(麻省总医院蒙根研究所);Division of Gastroenterology, Brigham and Women’s Hospital(布里洛妇女医院胃肠病科);Department of Medicine, Brigham and Women’s Hospital(布里洛妇女医院内科部);Department of Medicine, Massachusetts General Hospital(麻省总医院内科部);Department of Pathology, Massachusetts General Hospital(麻省总医院病理学部);Department of Health Humanities and Bioethics, University of Rochester School of Medicine and Dentistry(罗切斯特大学医学院和牙科学院健康人文与生物伦理学部);Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University(哈佛大学凯普纳人工智能研究所);Center for the History of Medicine, Countway Library of Medicine, Harvard Medical School(哈佛医学院医学史中心,考特维图书馆);Department of Global Health and Social Medicine, Harvard Medical School(哈佛医学院全球健康与社会医学部);Division of Pulmonary and Critical Care Medicine, Brigham and Women’s Hospital(布里洛妇女医院呼吸科和重症医学科)
提出 Dr. CaBot 代理 AI 系统,通过生成基于初始病例描述的幻灯片演示来模拟专家诊断推理,并在 NEJM CPC 和 NIH 未诊断疾病网络病例上取得优于前沿模型的表现,同时发布 CPC-Bench 基准以促进临床 AI 发展。
Teaching LLM to Reason: Reinforcement Learning from Algorithmic Problems without Code
University of Science and Technology of China(中国科学技术大学);Alibaba Group(阿里巴巴集团);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Teaching LLMs for Step-Level Automatic Math Correction via Reinforcement Learning
School of Computer Science and Technology, East China Normal University(华东师范大学计算机科学与技术学院);Lab of AI for Education, East China Normal University(华东师范大学教育人工智能实验室)