Reinforcement Learning from Verifiable Rewards (RLVR) has recently become a key paradigm for improving the reasoning abilities of Large Language Models (LLMs), yet it remains limited by sparse binary rewards and its ignorance of model-internal uncertainty. In this paper, we propose ConSteer-RL, a simple yet effective framework that integrates token-level confidence signals derived from model log-probabilities into RLVR training. Specifically, building upon the Group Relative Policy Optimization (GRPO) framework, we construct a confidence-aware reward by aggregating per-token probabilities into a scalar confidence score and incorporating it into an awareness-based reward shaping mechanism that penalizes overconfident errors while reinforcing correct and confident reasoning. Experimental results demonstrate that ConSteer-RL consistently outperforms strong GRPO baselines, achieving average improvements of 2.3%-4.0% across different model scales.
Latent space bias directions in LLMs capture confidence, not fairness
Institute of Astronomy & Kavli Institute for Cosmology, University of Cambridge（剑桥大学天文学研究所及卡夫利宇宙学研究所）；Risk and Security AI Lab, Visa Inc.（Visa公司风险与安全人工智能实验室）
Does higher interpretability imply better utility? A Pairwise Analysis on Sparse Autoencoders
School of Computing and Data Science, The University of Hong Kong（计算与数据科学学院，香港大学）；School of Data Science, The Chinese University of Hong Kong, Shenzhen（数据科学学院，香港中文大学（深圳））
How Post-Training Reshapes LLMs: A Mechanistic View on Knowledge, Truthfulness, Refusal, and Confidence
University of California, Los Angeles（加州大学洛杉矶分校）；University of Alberta（阿尔伯塔大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Harvard University（哈佛大学）
Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；Tsinghua University（清华大学）；The Chinese University of Hong Kong（香港中文大学）；Zhongguancun Academy（中关村学院）；Shenzhen Loop Area Institute（深圳河套学院）
