Sorbonne Université（索邦大学）；CNRS（法国国家科学研究中心）；LIP6（巴黎第六大学计算机科学实验室）；Carney Institute for Brain Science, Brown University（布朗大学卡尼脑科学研究所）；Idiap Research Institute（伊迪亚普研究所）；Australian Institute for Machine Learning, University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）
We introduce an evaluation methodology for visual question answering (VQA) to better diagnose cases of shortcut learning. These cases happen when a model exploits spurious statistical regularities to produce correct answers but does not actually deploy the desired behavior. There is a need to identify possible shortcuts in a dataset and assess their use before deploying a model in the real world. The research community in VQA has focused exclusively on question-based shortcuts, where a model might, for example, answer "What is the color of the sky" with "blue" by relying mostly on the question-conditional training prior and give little weight to visual evidence. We go a step further and consider multimodal shortcuts that involve both questions and images. We first identify potential shortcuts in the popular VQA v2 training set by mining trivial predictive rules such as co-occurrences of words and visual elements. We then introduce VQA-CounterExamples (VQA-CE), an evaluation protocol based on our subset of CounterExamples i.e. image-question-answer triplets where our rules lead to incorrect answers. We use this new evaluation in a large-scale study of existing approaches for VQA. We demonstrate that even state-of-the-art models perform poorly and that existing techniques to reduce biases are largely ineffective in this context. Our findings suggest that past work on question-based biases in VQA has only addressed one facet of a complex issue. The code for our method is available at https://github.com/cdancette/detect-shortcuts.
Seeing Time: Benchmarking Chronological Reasoning and Shortcut Biases in Vision-Language Models
College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Computing Technologies, RMIT University（皇家墨尔本理工学院计算技术学院）
Greedy Gradient Ensemble for Robust Visual Question Answering
Key Lab of Intell. Info. Process., Inst. of Comput. Tech., CAS（中国科学院计算技术研究所智能信息处理重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Kingsoft Cloud（金山云）；Peng Cheng Laboratory（鹏城实验室）；Huawei Technologies（华为技术有限公司）
PrinciplismQA: A Philosophy-Grounded Approach to Assessing LLM-Human Clinical Medical Ethics Alignment
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；National Health Data Institute, Shenzhen（深圳国家健康数据研究院）；Shenzhen Research Institute of Big Data（深圳大数据研究院）
Language Prior Is Not the Only Shortcut: A Benchmark for Shortcut Learning in VQA
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；Tencent Inc（腾讯公司）；Peking University（北京大学）
Improving Robustness to Multiple Spurious Correlations by Multi-Objective Optimization
POSTECH（浦项科技大学）；Department of Computer Science and Engineering, POSTECH（浦项科技大学计算机科学与工程学院）；Graduate School of Artificial Intelligence, POSTECH（浦项科技大学人工智能研究生院）
Look Beyond Bias with Entropic Adversarial Data Augmentation
Univ Lyon（里昂大学）；Ecole Centrale de Lyon（里昂中央理工学院）；CNRS（法国国家科学研究中心）；INSA Lyon（里昂国立应用科学学院）；Univ Claude Bernard Lyon 1（里昂第一大学）；Univ Louis Lumière Lyon 2（里昂第二大学）；LIRIS（里昂信息与图像研究所）；Thales LAS France（法国泰雷兹激光系统公司）
Revealing Model Biases: Assessing Deep Neural Networks via Recovered Sample Analysis
University of Tehran（德黑兰大学）；University of Bremen（不来梅大学）；Okinawa Institute of Science and Technology (OIST)（冲绳科学技术大学院大学）；Institute For Research In Fundamental Sciences (IPM)（基础科学研究所）
Towards Multimodal Sentiment Analysis Debiasing via Bias Purification
Academy for Engineering & Technology, Fudan University（复旦大学工程与技术学院）；Cognition and Intelligent Technology Laboratory (CIT Lab)（认知与智能技术实验室）；Tencent Youtu Lab（腾讯优图实验室）
