Existing Visual Question Answering (VQA) methods tend to exploit dataset biases and spurious statistical correlations, instead of producing right answers for the right reasons. To address this issue, recent bias mitigation methods for VQA propose to incorporate visual cues (e.g., human attention maps) to better ground the VQA models, showcasing impressive gains. However, we show that the performance improvements are not a result of improved visual grounding, but a regularization effect which prevents over-fitting to linguistic priors. For instance, we find that it is not actually necessary to provide proper, human-based cues; random, insensible cues also result in similar improvements. Based on this observation, we propose a simpler regularization scheme that does not require any external annotations and yet achieves near state-of-the-art performance on VQA-CPv2.
Why Does Grounding Hurt Medical VQA? Benchmarking, Diagnosis, and Fine-Tuning of Vision-Language Models
New York University, New York, USA（纽约大学）；Tsinghua University, Beijing, China（清华大学）；Columbia University, New York, USA（哥伦比亚大学）；University of Michigan, Ann Arbor, USA（密歇根大学）
CoTBox-TTT: Grounding Medical VQA with Visual Chain-of-Thought Boxes During Test-time Training
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Data Science, The Chinese University of Hong Kong, Shenzhen（香港大学深圳研究院数据科学学院）
Beyond Accuracy: Evaluating Visual Grounding In Multimodal Medical Reasoning
The University of Texas MD Anderson Cancer Center（德克萨斯大学MD安德森癌症中心）；Cohere Labs（Cohere实验室）；Eisai Inc.（艾伯维公司）；Indian Institute of Science, Bangalore（班加罗尔印度科学研究院）
Scalable Training of Spatially Grounded 2D Vision-Language Models for Radiology
Computer Vision Group, University of Freiburg, Germany（德国弗莱堡大学计算机视觉组）；Department of Radiology, Medical Center -- University of Freiburg, Germany（德国弗莱堡大学医学中心放射科）；CRIION-AI Lab, Freiburg, Germany（德国弗莱堡CRIION-AI实验室）
