While Vision-Language Models (VLMs) are increasingly used to generate reward signals for training embodied agents to follow instructions, our research reveals that agents guided by VLM rewards often underperform compared to those employing only intrinsic (exploration-driven) rewards, contradicting expectations set by recent work. We hypothesize that false positive rewards -- instances where unintended trajectories are incorrectly rewarded -- are more detrimental than false negatives. Our analysis confirms this hypothesis, revealing that the widely used cosine similarity metric is prone to false positive reward estimates. To address this, we introduce BiMI ({Bi}nary {M}utual {I}nformation), a novel reward function designed to mitigate noise. BiMI significantly enhances learning efficiency across diverse and challenging embodied navigation environments. Our findings offer a nuanced understanding of how different types of reward noise impact agent learning and highlight the importance of addressing multimodal reward signal noise when training embodied agents
Harnessing VLM in Embodied RL: From Reward Regressor to Success Detector via Exploration-Augmented Key-Observation Gating
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所 多模态人工智能系统国家重点实验室）；School of Artificial Intelligence, University of the Chinese Academy of Sciences（中国科学院大学 人工智能学院）；AGIBOT
MARVL: Multi-Stage Guidance for Robotic Manipulation via Vision-Language Models
School of Intelligent Science and Technology, Nanjing University, China（南京大学智能科学与技术学院）；National Key Laboratory for Novel Software Technology, School of Artificial Intelligence, Nanjing University, China（南京大学新型软件技术国家实验室，人工智能学院）；School of Artificial Intelligence, Nanjing University, China（南京大学人工智能学院）；MACS Lab, University of Washington（华盛顿大学MACS实验室）
The Horcrux: Mechanistically Interpretable Task Decomposition for Detecting and Mitigating Reward Hacking in Embodied AI Systems
Berkeley AI Safety Initiative (BASIS) UC Berkeley（伯克利人工智能安全计划（BASIS）伯克利大学）；Department of Electrical and Computer Engineering Johns Hopkins University（电气与计算机工程系约翰霍普金斯大学）
DynaCF: Mitigating Shortcut Learning in Reward Models via Dynamic Counterfactual Sensitivity
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；New Jersey Institute of Technology（新泽西理工学院）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）
MVR: Multi-view Video Reward Shaping for Reinforcement Learning
School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）；State Key Laboratory of General Artificial Intelligence, BIGAI（通用人工智能国家重点实验室）
GD$^2$PO: Mitigating Multi-Reward Conflicts via Group-Dynamic reward-Decoupled Policy Optimization
Qwen Large Model Application Team, Alibaba（阿里巴巴通义千问大模型应用团队）；Renmin University of China（中国人民大学）；Peking University（北京大学）；ETH Zürich（苏黎世联邦理工学院）；University of Zurich（苏黎世大学）；The Chinese University of Hong Kong（香港中文大学）
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University（北京未来区块链与隐私计算先进创新中心，人工智能学院，北京航空航天大学）；Tsinghua University（清华大学）
针对准则奖励中因忽略准则间依赖关系导致的虚假信用传播问题，提出概率图框架Graphical Event Aggregation for Rubric rewards (GEAR)，通过建模潜在伯努利事件和软抑制传播实现依赖感知的奖励聚合，在多个基准上提升性能并减少信用泄漏。
Understanding and Mitigating Token-Pruning-Induced Vulnerabilities in VLMs
University of Electronic Science and Technology of China（电子科技大学）；Southwestern University of Finance and Economics（西南财经大学）；Tongji University（同济大学）；Shanghai Innovation Institute（上海创新研究院）
Mitigating Reward Over-Optimization in RLHF via Behavior-Supported Regularization
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；The State Key Lab of Brain-Machine Intelligence, Zhejiang University（浙江大学脑机智能国家重点实验室）；LLM Safety Centre, Beijing Academy of Artificial Intelligence（北京人工智能研究院大语言模型安全中心）；Center for AI Safety and Governance, Peking University（北京大学人工智能安全与治理中心）
Mitigating Reward Over-optimization in Direct Alignment Algorithms with Importance Sampling
VinUniversity（文大学）；VinUni-Illinois Smart Health Center（VinUni-伊利诺伊智能健康中心）；University of Stuttgart（斯图加特大学）；University of California Los Angeles (UCLA)（加州大学洛杉矶分校）；International University - VNUHCM（国际大学 - VNUHCM）；University of Science - VNUHCM（科学大学 - VNUHCM）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Oldenburg University（奥尔登堡大学）；Max Planck Research School for Intelligent Systems (IMPRS-IS)（马克斯·普朗克智能系统研究学校）
