This paper examines a phenomenon in multimodal language models where pre-marked options in question images can significantly influence model responses. Our study employs a systematic methodology to investigate this effect: we present models with images of multiple-choice questions, which they initially answer correctly, then expose the same model to versions with pre-marked options. Our findings reveal a significant shift in the models' responses towards the pre-marked option, even when it contradicts their answers in the neutral settings. Comprehensive evaluations demonstrate that this agreeableness bias is a consistent and quantifiable behavior across various model architectures. These results show potential limitations in the reliability of these models when processing images with pre-marked options, raising important questions about their application in critical decision-making contexts where such visual cues might be present.
ReadBench: Measuring the Dense Text Visual Reading Ability of Vision-Language Models
Artificial Intelligence and Intelligent Information Systems, University of Trier（人工智能与智能信息系统，特里尔大学）；German Research Center for Artificial Intelligence (DFKI), Trier（德国人工智能研究中心（DFKI），特里尔）
Measuring Social Bias in Vision-Language Models with Face-Only Counterfactuals from Real Photos
School of Intelligence Science and Engineering, Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳）智能科学与工程学院）；School of Information Science and Engineering, Ningbo University（宁波大学信息科学与工程学院）；School of Information Science and Technology, University of Science and Technology of China（中国科学技术大学信息科学与技术学院）
Many Dialects, Many Languages, One Cultural Lens: Evaluating Multilingual VLMs for Bengali Culture Understanding Across Historically Linked Languages and Regional Dialects
United International University（国际联合大学）；BRAC University（布拉塔克大学）；University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）
When Meaning Travels: A Granular Lens on Hybrid-MoE's Role in Idiomatic Understanding for Language Models
Department of Computer Science and Engineering, Indian Institute of Technology Patna, India（印度理工学院帕纳瓦分校计算机科学与工程系）；School of Information Technology, King Mongkut’s Institute of Technology Ladkrabang, Thailand（泰国拉差班国王理工大学信息科技学院）
GDCNet: Generative Discrepancy Comparison Network for Multimodal Sarcasm Detection
State Key Laboratory of AI Safety, Institute of Computing Technology (ICT), Chinese Academy of Sciences (CAS)（人工智能安全国家重点实验室，计算技术研究所（ICT），中国科学院（CAS））；University of Chinese Academy of Sciences, CAS（中国科学院大学）；Pengcheng Laboratory（鹏城实验室）；Shenzhen Stock Exchange（深圳证券交易所）
