Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities in multimodal reasoning. However, prevailing reinforcement learning (RL) paradigms lack explicit counterfactual enhancement and causal learning mechanisms. This fundamental deficiency results in severe grounding failures, manifesting as a tendency to ignore visual evidence in favor of language priors or exhibiting hallucination drift during long chain-of-thought reasoning. To address this root cause, we propose CounterFactual Policy Optimization (CFPO), a novel framework that enforces causal consistency between visual perception and textual reasoning. CFPO introduces a cross-modal counterfactual enhancement mechanism, which regularizes the policy by maximizing the discrepancy between the model's predictions and those from a counterfactual state where critical visual cues are suppressed. This approach seamlessly integrates with standard algorithms like GRPO and DAPO without requiring external reward models or additional supervision. Extensive experiments demonstrate that CFPO significantly improves reasoning fidelity, achieving consistent gains of 3.17%-6.25% over standard RL baselines and 1.32%-2.13% over the state-of-the-art perception-aware method (PAPO). Code is available at https://github.com/Raven-July/CFPO.
A$^2$Safe: Counterfactual Evidence-Aligned Adaptive Agent Collaboration for Safe and Effective Visual Question Answering
Macau University of Science and Technology(澳门科技大学);Wuhan University of Technology(武汉理工大学);Nanjing Institute of Technology(南京工程学院);National Tsing Hua University(国立清华大学)
Measuring Social Bias in Vision-Language Models with Face-Only Counterfactuals from Real Photos
School of Intelligence Science and Engineering, Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳)智能科学与工程学院);School of Information Science and Engineering, Ningbo University(宁波大学信息科学与工程学院);School of Information Science and Technology, University of Science and Technology of China(中国科学技术大学信息科学与技术学院)
Faithful-MR1: Faithful Multimodal Reasoning via Anchoring and Reinforcing Visual Attention
AMAP, Alibaba Group(阿里云实验室,阿里巴巴集团);University of Chinese Academy of Sciences(中国科学院大学);Tsinghua University(清华大学);Nanyang Technological University(南洋理工大学)
Preferred Networks;The University of Tokyo(东京大学);RIKEN(理化学研究所)
This study addresses generating counterfactual explanations with multimodal information. Our goal is not only to classify a video into a specific category, but also to provide explanations on why it is not categorized to a specific class with combinations of visual-linguistic information. Requirements that the expected output should satisfy are referred to as counterfactuality in this paper: (1) Compatibility of visual-linguistic explanations, and (2) Positiveness/negativeness for the specific positive/negative class. Exploiting a spatio-temporal region (tube) and an attribute as visual and li…
GenderBias-\emph{VL}: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probing
Beihang University(北京航空航天大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);National University of Singapore(新加坡国立大学);Nanyang Technological University(南洋理工大学)
ACQUIRED: A Dataset for Answering Counterfactual Questions In Real-Life Videos
University of Maryland, College Park(马里兰大学帕克分校);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);Information Sciences Institute, University of Southern California(南加州大学信息科学研究所)
Towards Multimodal Sentiment Analysis Debiasing via Bias Purification
Academy for Engineering & Technology, Fudan University(复旦大学工程与技术学院);Cognition and Intelligent Technology Laboratory (CIT Lab)(认知与智能技术实验室);Tencent Youtu Lab(腾讯优图实验室)
Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);Nanyang Technological University(南洋理工大学);Tsinghua University(清华大学)
Beyond Accuracy: Evaluating Visual Grounding In Multimodal Medical Reasoning
The University of Texas MD Anderson Cancer Center(德克萨斯大学MD安德森癌症中心);Cohere Labs(Cohere实验室);Eisai Inc.(艾伯维公司);Indian Institute of Science, Bangalore(班加罗尔印度科学研究院)
A Pathway to General-Purpose Scientific AI: Multimodal Comprehension of Scientific Images
TIB Leibniz Information Centre for Science and Technology(莱布尼茨科学与技术信息中心(TIB));Eindhoven University of Technology(埃因霍温理工大学);Freie Universität Berlin(柏林自由大学);International Center for Chemical and Biological Sciences, University of Karachi(卡拉奇大学国际化学与生物科学中心);National University of Sciences and Technology(国家科技大学);University of Warwick(华威大学);PSG College of Technology(PSG理工学院);Aalto University(阿尔托大学)
TwinICL: Diagnosing Multimodal In-Context Learning through Paired Counterfactuals
University of California, Los Angeles(加州大学洛杉矶分校);National Taiwan University(国立台湾大学);Texas A&M University(德克萨斯A&M大学);NVIDIA AI Technology Center(英伟达AI技术中心);Arena Intelligence Inc(Arena Intelligence公司)
Mitigating Group-Level Fairness Disparities in Federated Visual Language Models
Great Bay University(大湾大学);Tsinghua Shenzhen International Graduate School(清华大学深圳国际研究生院);Nanyang Technological University(南洋理工大学);Beijing University of Posts and Telecommunications(北京邮电大学);Shenzhen University(深圳大学);Pengcheng Laboratory(鹏城实验室);Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区)