Pre-trained Vision-Language Models (VLMs) are able to understand visual concepts, describe and decompose complex tasks into sub-tasks, and provide feedback on task completion. In this paper, we aim to leverage these capabilities to support the training of reinforcement learning (RL) agents. In principle, VLMs are well suited for this purpose, as they can naturally analyze image-based observations and provide feedback (reward) on learning progress. However, inference in VLMs is computationally expensive, so querying them frequently to compute rewards would significantly slowdown the training of an RL agent. To address this challenge, we propose a framework named Code as Reward (VLM-CaR). VLM-CaR produces dense reward functions from VLMs through code generation, thereby significantly reducing the computational burden of querying the VLM directly. We show that the dense rewards generated through our approach are very accurate across a diverse set of discrete and continuous environments, and can be more effective in training RL policies than the original sparse environment rewards.
From Perception to Punchline: Empowering VLM with the Art of In-the-wild Meme
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；School of Software Engineering（软件工程学院）；Columbia Engineering（哥伦比亚工程学院）；Columbia University（哥伦比亚大学）；The Chinese University of Hong Kong MMLab（香港中文大学MMLab）
Harnessing VLM in Embodied RL: From Reward Regressor to Success Detector via Exploration-Augmented Key-Observation Gating
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所 多模态人工智能系统国家重点实验室）；School of Artificial Intelligence, University of the Chinese Academy of Sciences（中国科学院大学 人工智能学院）；AGIBOT
CheXPO-v2: Preference Optimization for Chest X-ray VLMs with Knowledge Graph Consistency
School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）；Warren Alpert Medical School of Brown University（布朗大学沃伦·阿尔珀特医学学院）；School of Biomedical Engineering, Capital Medical University（首都医科大学生物医学工程学院）
BudgetThinker: Empowering Budget-aware LLM Reasoning with Control Tokens
Institute for AI Industry Research (AIR) Tsinghua University（人工智能产业研究院（AIR）清华大学）；Global Innovation Exchange & Department of Automation Tsinghua University（全球创新交流中心及自动化系 清华大学）
ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学人工智能学院 Gallagher 分校）；Baidu Inc., Beijing, China（百度公司，北京，中国）；Carnegie Mellon University（卡内基梅隆大学）
Empowering VLMs for Few-Shot Multimodal Time Series Classification via Tailored Agentic Reasoning
Sun Yat-sen University（中山大学）；Xiaomi Corporation（小米公司）；University of Science and Technology of China（中国科学技术大学）；National University of Singapore（新加坡国立大学）
MARVL: Multi-Stage Guidance for Robotic Manipulation via Vision-Language Models
School of Intelligent Science and Technology, Nanjing University, China（南京大学智能科学与技术学院）；National Key Laboratory for Novel Software Technology, School of Artificial Intelligence, Nanjing University, China（南京大学新型软件技术国家实验室，人工智能学院）；School of Artificial Intelligence, Nanjing University, China（南京大学人工智能学院）；MACS Lab, University of Washington（华盛顿大学MACS实验室）
MVR: Multi-view Video Reward Shaping for Reinforcement Learning
School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）；State Key Laboratory of General Artificial Intelligence, BIGAI（通用人工智能国家重点实验室）
Learning to Reason About Implicit Multimodal Harm: From Diagnosis to Generalization
Max Planck Institute for Informatics（马克斯·普朗克院信息研究所）；Saarland Informatics Campus（萨尔兰州信息校园）；Saarland University（萨尔兰州大学）；The University of Edinburgh（爱丁堡大学）；Samsung AI Center, Cambridge（三星AI中心，剑桥）
ThinkJEPA: Empowering Latent World Models with Large Vision-Language Reasoning Model
Northeastern University（东北大学）；University of California San Diego（加州大学圣地亚哥分校）；University of Maryland（马里兰大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Washington（华盛顿大学）
MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?
University of Chicago（芝加哥大学）；NUS（新加坡国立大学）；Stanford University（斯坦福大学）；UCSC（加州大学圣克鲁兹分校）；Duke University（杜克大学）；ESSEC（埃塞克高等商学院）；Illinois Tech（伊利诺伊理工学院）；UCSD（加州大学圣地亚哥分校）；Peking University（北京大学）；University of Queensland（昆士兰大学）；Stony Brook University（石溪大学）；UNC-Chapel Hill（北卡罗来纳大学教堂山分校）；USTC（中国科学技术大学）
