ATTNPO: Attention-Guided Process Supervision for Efficient Reasoning
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院);Baidu Inc.(百度公司);Tianjin University(天津大学)
Shuaiyi Nie, Siyu Ding, Wenyuan Zhang, Linhao Yu, Tianmeng Yang, Yao Chen, Weichong Yin, Yu Sun, Hua Wu, Tingwen Liu
Large reasoning models trained with reinforcement learning and verifiable rewards (RLVR) achieve strong performance on complex reasoning tasks, yet often overthink, generating redundant reasoning without performance gains. Existing trajectory-level length penalties often fail to effectively shorten reasoning length and degrade accuracy, as they uniformly treat all reasoning steps and lack fine-grained signals to distinguish redundancy from necessity. Meanwhile, process-supervised methods are typically resource-intensive and suffer from inaccurate credit assignment. To address these issues, we propose ATTNPO, a low-overhead process-supervised RL framework that leverages the model's intrinsic attention signals for step-level credit assignment. We first identify a set of special attention heads that naturally focus on essential steps while suppressing redundant ones. By leveraging the attention scores of these heads, We then employ two sub-strategies to mitigate overthinking by discouraging redundant steps while preserving accuracy by reducing penalties on essential steps. Experimental results show that ATTNPO substantially reduces reasoning length while significantly improving performance across 9 benchmarks.
Rewarding Novel Deductions: Solver-guided Process Supervision for Logical Reasoning
Information Technology University(信息技术大学);Huazhong Agricultural University(华中农业大学);Qatar Computing Research Institute, Hamad Bin Khalifa University(卡塔尔计算研究所,哈马德·本·哈利法大学)
From Anchors to Supervision: Memory-Graph Guided Corpus-Free Unlearning for Large Language Models
College of Computer Science and Artificial Intelligence, Fudan University(复旦大学计算机科学与人工智能学院);Shanghai University of Electric Power(上海电力大学);School of Information Science and Engineering, East China University of Science and Technology(东华大学信息科学与工程学院)
Enhancing Logical Reasoning in Language Models via Symbolically-Guided Monte Carlo Process Supervision
School of Computer Science, University of Sheffield(谢菲尔德大学计算机科学学院);School of Electronic Engineering and Computer Science, Queen Mary University of London(伦敦大学玛丽女王学院电子工程与计算机科学学院);The Alan Turing Institute(艾伦·图灵研究所)
Teacher-Guided Tool Learning for Small Language Models via Bidirectional KL Constraints
Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Security Capability Center(360安全能力中心);Nanjing University(南京大学);Tongji University(同济大学);Shanghai Jiao Tong University(上海交通大学)
Boosting Medical Image-based Cancer Detection via Text-guided Supervision from Reports
DAMO Academy, Alibaba Group(阿里巴巴集团达摩学院);Hefei Comprehensive National Science Center(合肥综合性国家科学中心);Hupan Lab(湖畔实验室);Sun Yat-sen University Cancer Center(中山大学肿瘤防治中心);South China Hospital, Medical School, Shenzhen University(深圳大学医学院华南医院)