We introduce Nemotron-Cascade 2, an open 30B MoE model with 3B activated parameters that delivers best-in-class reasoning and strong agentic capabilities. Despite its compact size, its mathematical and coding reasoning performance approaches that of frontier open models. It is the second open-weight LLM, after DeepSeekV3.2-Speciale-671B-A37B, to achieve Gold Medal-level performance in the 2025 International Mathematical Olympiad (IMO), the International Olympiad in Informatics (IOI), and the ICPC World Finals, demonstrating remarkably high intelligence density with 20x fewer parameters. In contrast to Nemotron-Cascade 1, the key technical advancements are as follows. After SFT on a meticulously curated dataset, we substantially expand Cascade RL to cover a much broader spectrum of reasoning and agentic domains. Furthermore, we introduce multi-domain on-policy distillation from the strongest intermediate teacher models for each domain throughout the Cascade RL process, allowing us to efficiently recover benchmark regressions and sustain strong performance gains along the way. We release the collection of model checkpoint and training data.
From Deferral to Learning: Online In-Context Knowledge Distillation for LLM Cascades
Department of XXX, University of YYY, Location, Country(XXX系,YYY大学,地点,国家);School of ZZZ, Institute of WWW, Location, Country(ZZZ学院,WWW研究所,地点,国家);Department of Electrical and Computer Engineering, Rutgers University, NJ, USA(电气与计算机工程系,Rutgers大学,新泽西州,美国);Computer Engineering Department, University of Illinois Chicago, Illinois, USA(计算机工程系,伊利诺伊大学芝加哥分校,伊利诺伊州,美国)
Backtracking When It Strays: Mitigating Dual Exposure Biases in LLM Reasoning Distillation
College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院);Key Laboratory of Symbolic Computation and Knowledge Engineering, MoE, Jilin University(吉林大学符号计算与知识工程重点实验室);Tongyi Lab, Alibaba Group(阿里集团通义实验室);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);School of Artificial Intelligence, Jilin University(吉林大学人工智能学院)
Improve Student's Reasoning Generalizability through Cascading Decomposed CoTs Distillation
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
SOD: Step-wise On-policy Distillation for Small Language Model Agents
Zhejiang University(浙江大学);Large Language Model Department, Tencent(腾讯大语言模型部门);University of Science and Technology of China(中国科学技术大学);National University of Singapore(新加坡国立大学)
BLSP-KD: Bootstrapping Language-Speech Pre-training via Knowledge Distillation
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Alibaba Group(阿里巴巴集团);Machine Intelligence Technology Lab, Alibaba Group(阿里巴巴集团机器智能技术实验室)