Physical reasoning over images requires integrating perception, domain knowledge, and multi-step symbolic inference, yet Vision-Language Models (VLMs) fall far short of human performance on physics benchmarks. Reinforcement learning can improve reasoning via reward signals, but it remains unclear whether these signals must come from outside the model or can be derived from the model itself. We compare external rewards, which rely on an outside verifier (rule-based format and answer-correctness checks, or an LLM-judge rubric), with an internal attention-based reward computed from the model's own attention over image regions, requiring no labels or judge. We find that external rewards drive most of the accuracy gains, while preserving more coherent reasoning when rewarded for reasoning tokens. The internal attention-based reward, provides an annotation-free grounding signal that improves spatial-relation reasoning and is complementary to external rewards: adding it to format and accuracy rewards, at no additional annotation cost, improves average accuracy on Molmo~2 while generalizing to out-of-distribution tasks. Our results suggest that internal attention rewards can supplement external supervision and lead to improved physical reasoning.
Disentangling Perception and Reasoning in Multimodal LLMs via Reward Design
Department of Computer Science, Dartmouth College(达特茅斯大学计算机科学系);Department of Computer Science, University of Central Florida(中央佛罗里达大学计算机科学系);Independent Researcher(独立研究者)
PACR: Progressively Ascending Confidence Reward for LLM Reasoning
Korea Advanced Institute of Science and Technology (KAIST)(韩国科学技术院);Microsoft Research Asia (MSRA)(微软亚洲研究院);University of Illinois at Urbana-Champaign (UIUC)(伊利诺伊大学厄巴纳-香槟分校)
University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of California, San Diego(加州大学圣地亚哥分校);Texas A&M University(德克萨斯A&M大学);Stevens Institute of Technology(史蒂文斯理工学院)
Metacognition as Reward: Reinforcing LLM Reasoning via Knowledge and Regulation Signals
Tongji University(同济大学);Shanghai AI Laboratory(上海人工智能实验室);Nanyang Technological University(南洋理工大学);University of Science and Technology of China(中国科学技术大学);EPFL(苏黎世联邦理工学院);Wuhan University(武汉大学)
InfoDensity: Rewarding Information-Dense Traces for Efficient Reasoning
Institute for Infocomm Research (I 2 R), A*STAR, Singapore(信息与通信研究机构(I 2 R),A*STAR,新加坡);Centre for Frontier AI Research (CFAR), A*STAR, Singapore(前沿人工智能研究中心(CFAR),A*STAR,新加坡)
Towards Hierarchical Multi-Step Reward Models for Enhanced Reasoning in Large Language Models
the University of Hong Kong(香港大学);Tsinghua University(清华大学);Georgia Institute of Technology(佐治亚理工学院);National University of Singapore(新加坡国立大学);Zhejiang University(浙江大学);Renmin University of China(中国人民大学);the Chinese University of Hong Kong(香港中文大学)
FlowRL: Matching Reward Distributions for LLM Reasoning
Shanghai Jiao Tong University(上海交通大学);Shanghai AI Laboratory(上海人工智能实验室);Microsoft Research(微软研究院);Tsinghua University(清华大学);Peking University(北京大学);Renmin University of China(中国人民大学);Stanford University(斯坦福大学);Toyota Technological Institute at Chicago(芝加哥丰田技术研究所)
BeamPERL: Parameter-Efficient RL with Verifiable Rewards Specializes Compact LLMs for Structured Beam Mechanics Reasoning
Department of Mechanical Engineering(机械工程系);Massachusetts Institute of Technology(麻省理工学院);Department of Civil and Environmental Engineering(土木与环境工程系);Schwarzman College of Computing(施瓦茨曼计算学院)
Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Peng Cheng Laboratory(鹏城实验室);Wuhan AI Research(武汉人工智能研究院)
ProcessThinker: Enhancing Multi-modal Large Language Models Reasoning via Rollout-based Process Reward
LMU Munich(慕尼黑大学);Harvard University(哈佛大学);University of Cambridge(剑桥大学);Mina AI;Konrad Zuse School of Excellence in Reliable AI (relAI)(康拉德·楚泽可靠人工智能卓越学校(relAI))
Posterior-GRPO: Rewarding Reasoning Processes in Code Generation
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);The State Key Laboratory of Blockchain and Data Security, Zhejiang University(浙江大学区块链与数据安全国家重点实验室);Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security(杭州高新技术区(滨江)区块链与数据安全研究院)
Socratic-PRMBench: Benchmarking Process Reward Models with Systematic Reasoning Patterns
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Tongyi Lab, Alibaba Group(阿里云实验室)
Does LLM Alignment Really Need Diversity? An Empirical Study of Adapting RLVR Methods for Moral Reasoning
Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);Microsoft Research(微软研究院);University of Michigan(密歇根大学);Shanghai Jiao Tong University(上海交通大学);CUHKSZ(香港中文大学(深圳));THU(清华大学)
Learning to Reason About Implicit Multimodal Harm: From Diagnosis to Generalization
Max Planck Institute for Informatics(马克斯·普朗克院信息研究所);Saarland Informatics Campus(萨尔兰州信息校园);Saarland University(萨尔兰州大学);The University of Edinburgh(爱丁堡大学);Samsung AI Center, Cambridge(三星AI中心,剑桥)
Latent Reward Steering: An Adaptive Inference-Time Framework that Implicitly Promotes Cognitive Behaviors in Reasoning LLMs
Rutgers University(罗格斯大学);South China Agricultural University(华南农业大学);Columbia University(哥伦比亚大学);Fenz.AI;QuantaAlpha;Adobe;Santa Clara University(圣克拉拉大学);City University of Hong Kong(香港城市大学)
Think, then Score: Decoupled Reasoning and Scoring for Video Reward Modeling
University of Science and Technology of China(中国科学技术大学);Kling Team, Kuaishou Technology(快手科技 Kling 团队);Institute of Software Chinese Academy of Sciences(中国科学院软件研究所)
Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward
University of Science and Technology of China(中国科学技术大学);State Key Laboratory of Cognitive Intelligence(认知智能国家重点实验室);iFLYTEK AI Research(iFLYTEK人工智能研究院);Hong Kong University of Science and Technology(香港科学与技术大学)