Recent advancements in Multimodal Large Language Models (MLLMs) have demonstrated impressive performance on standard visual reasoning benchmarks. However, there is growing concern that these models rely excessively on linguistic shortcuts rather than genuine visual grounding, a phenomenon we term Text Bias. In this paper, we investigate the fundamental tension between visual perception and linguistic priors. We decouple the sources of this bias into two dimensions: Internal Corpus Bias, stemming from statistical correlations in pretraining, and External Instruction Bias, arising from the alignment-induced tendency toward sycophancy. To quantify this effect, we introduce V-FAT (Visual Fidelity Against Text-bias), a diagnostic benchmark comprising 4,026 VQA instances across six semantic domains. V-FAT employs a Three-Level Evaluation Framework that systematically increases the conflict between visual evidence and textual information: (L1) internal bias from atypical images, (L2) external bias from misleading instructions, and (L3) synergistic bias where both coincide. We introduce the Visual Robustness Score (VRS), a metric designed to penalize "lucky" linguistic guesses and reward true visual fidelity. Our evaluation of 12 frontier MLLMs reveals that while models excel in existing benchmarks, they experience significant visual collapse under high linguistic dominance.
Benchmarking Language Modeling for Lossless Compression of Full-Fidelity Audio
University of California, San Diego, Computer Science and Engineering Department(加州大学圣地亚哥分校计算机科学与工程系);Carnegie Mellon University, School of Computer Science(卡内基梅隆大学计算机科学学院)
Generating Synthetic Electronic Health Record Data: a Methodological Scoping Review with Benchmarking on Phenotype Data and Open-Source Software
Department of Biostatistics, University of Michigan(生物统计学系,密歇根大学);Department of Biomedical Informatics and Data Science, Yale University(生物医学信息学与数据科学系,耶鲁大学);Department of Biostatistics, Yale University(生物统计学系,耶鲁大学)
AtomBench: A Benchmarking Framework for Generative Crystal Reconstruction Models in Conventional Superconductors
Department of Physics and Astronomy, West Virginia University(物理与天文学系,西弗吉尼亚大学);Department of Materials Science and Engineering, Whiting School of Engineering, The Johns Hopkins University(材料科学与工程系,韦廷工程学院,约翰霍普金斯大学)
MSRBench: A Benchmarking Dataset for Music Source Restoration
Independent Researcher(独立研究者);John Hopkins University(约翰·霍普金斯大学);The Chinese University of Hong Kong(香港中文大学);Sony AI(索尼人工智能);King's College London(伦敦国王学院)
ProMediConv: Benchmarking Proactive Conversational Agents in Legal Dispute Mediation
University of Science and Technology of China(中国科学技术大学);Singapore Management University(新加坡管理大学);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院)