School of Mathematical Sciences, Shanghai Jiao Tong University（上海交通大学数学科学学院）；School of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学人工智能学院）；Institute for Advanced Algorithms Research, Shanghai（上海高级算法研究院）
Since the inception of Large Language Models (LLMs), the quest to efficiently train them for superior reasoning capabilities has been a pivotal challenge. The dominant training paradigm for LLMs is based on next token prediction (NTP). Alternative methodologies, called Critical Token Prediction (CTP), focused exclusively on specific critical tokens (such as the answer in Q\&A dataset), aiming to reduce the overfitting of extraneous information and noise. Contrary to initial assumptions, our research reveals that despite NTP's exposure to noise during training, it surpasses CTP in reasoning ability. We attribute this counterintuitive outcome to the regularizing influence of noise on the training dynamics. Our empirical analysis shows that NTP-trained models exhibit enhanced generalization and robustness across various benchmark reasoning datasets, demonstrating greater resilience to perturbations and achieving flatter loss minima. These findings illuminate that NTP is instrumental in fostering reasoning abilities during pretraining, whereas CTP is more effective for finetuning, thereby enriching our comprehension of optimal training strategies in LLM development.
Not All Tokens Matter: Towards Efficient LLM Reasoning via Token Significance in Reinforcement Learning
Tsinghua University（清华大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Peking University（北京大学）；Qwen Large Model Application Team, Alibaba（阿里云文大模型应用团队）；Microsoft（微软）；Shenzhen Key Laboratory of Ubiquitous Data Enabling, Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院 ubiquitous 数据赋能重点实验室）
Reflection Steering: Disentangling Reflection from Reasoning in Activation Space for Token-Efficient Inference
School of Computing and Data Science, The University of Hong Kong（香港大学计算与数据科学学院）；Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算学院）；School of Data Science, Lingnan University（岭南大学数据科学学院）；Centre for Learning, Teaching and Technology, The Education University of Hong Kong（香港教育大学学习、教学与技术中心）
GMTS: Gradient Magnitude-based Token Selection Improves RLVR Training for LLM Reasoning
School of Mathematical Sciences, Shanghai Jiao Tong University（上海交通大学数学科学学院）；Institute of Natural Sciences, Shanghai Jiao Tong University（上海交通大学自然科学研究院）
HSE University（俄罗斯国立高等经济大学）；Lomonosov Moscow State University（莫斯科国立大学）；Harbin Institute of Technology（哈尔滨工业大学）；ISP RAS（俄罗斯科学院系统编程研究所）；RANEPA（俄罗斯总统国民经济与公共管理学院）；ITMO University（圣彼得堡国立信息技术机械与光学大学）；Skoltech（斯科尔科沃科学技术研究院）
On the Token Value Inequality in Efficient Reasoning
Purdue University（普渡大学）；Rochester Institute of Technology（罗切斯特理工学院）；MIT-IBM Watson AI Lab（MIT-IBM沃森人工智能实验室）；University of Missouri-Kansas City（密苏里大学堪萨斯城分校）；Rutgers University（罗格斯大学）；Meta AI
Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学北京校区人工智能学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University College London, University of London（伦敦大学大学学院）；Dalian University of Technology（大连理工大学）
PiERN: Token-Level Routing for Integrating High-Precision Computation and Reasoning
Peking University（北京大学）；Peking University Changsha Institute for Computing and Digital Economy（北京大学长沙计算与数字经济研究院）；Beihang University（北京航空航天大学）；Tsinghua University（清华大学）
Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuning
Southern University of Science and Technology（南方科技大学）；Peking University（北京大学）；Shanghai University of Finance and Economics（上海金融学院）；Tsinghua University（清华大学）
