Reinforcement Learning (RL) and Supervised Fine-Tuning (SFT) are the two dominant paradigms for enhancing Large Language Model (LLM) performance on downstream tasks. While RL generally preserves broader model capabilities (retention) better than SFT, it comes with significant costs: complex reward engineering, instability, and expensive on-policy sampling. In contrast, SFT is efficient but brittle, often suffering from catastrophic forgetting due to $\textbf{Supervision Mismatch}$: the divergence between the model's evolving policy and static training labels. We address this trade-off with $\textbf{Trajectory-Mixed Supervision (TMS)}$, a reward-free framework that approximates the on-policy benefits of RL by creating a dynamic curriculum from the model's own historical checkpoints. TMS minimizes $\textit{Policy-Label Divergence (PLD)}$, preventing the mode collapse that drives forgetting in standard SFT. Experiments across reasoning (MATH, GSM8K) and instruction-following benchmarks demonstrate that TMS effectively shifts the accuracy--retention Pareto frontier. While RL remains the gold standard for retention, TMS significantly outperforms standard and iterative SFT, bridging the gap to RL without requiring reward models or verifiers. Mechanistic analysis confirms that PLD drift accurately predicts forgetting and that TMS successfully mitigates this drift.
TSRL: Trigger-based Supervised Reinforcement Learning for Continual Agent Training
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；University of Science and Technology of China（中国科学技术大学）
Unlocking Data Value in Finance: A Study on Distillation and Difficulty-Aware Training
Shanghai Artificial Intelligence Laboratory, OpenDataLab, OpenDataArena（上海人工智能实验室、OpenDataLab、OpenDataArena）；Hong Kong University of Science and Technology（香港科技大学）
Scaling Near-Optimal SFT-RL Annotation Budget Allocation from Small to Large LLMs
National University of Singapore（新加坡国立大学）；Agency for Science, Technology, and Research (A*STAR)（新加坡科技研究局）；Singapore-MIT Alliance for Research and Technology Centre（新加坡-麻省理工研究与技术联盟中心）
RASFT: Rollout-Adaptive Supervised Fine-Tuning for Reasoning
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；New Jersey Institute of Technology（新泽西理工学院）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）
Learning in Context, Guided by Choice: A Reward-Free Paradigm for Reinforcement Learning with Transformers
Department of Electrical and Computer Engineering, Duke University, Durham, US（电子工程系，杜克大学，达勒姆，美国）；Department of Computer Science, Duke University, Durham, US（计算机科学系，杜克大学，达勒姆，美国）；Department of Biostatistics and Bioinformatics, Duke University, Durham, US（生物统计学与生物信息学系，杜克大学，达勒姆，美国）；Department of Statistics and Data Science, Yale University, New Haven, US（统计学与数据科学系，耶鲁大学，纽 Haven，美国）
Reinforcement Learning with Semantic Rewards Enables Low-Resource Language Expansion without Alignment Tax
Minzu University of China（中国民族大学）；Ant Group（蚂蚁集团）；Shanghai Jiao Tong University（上海交通大学）；University of Macau（澳门大学）；Peking University（北京大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Hainan International College, Minzu University of China（中国民族大学海南国际学院）
