一旦正确，仍错误：多语言视觉-语言模型中的反事实幻觉
Once Correct, Still Wrong: Counterfactual Hallucination in Multilingual Vision-Language Models
AI总结：
研究探讨多语言视觉-语言模型在文化背景下反事实幻觉的问题，提出M²CQA基准测试，通过17个中东国家图像和多语言对比陈述评估模型性能，发现阿拉伯语尤其在方言中反事实幻觉率显著上升。
摘要：
视觉-语言模型（VLMs）在高精度的同时仍可能接受文化上合理但视觉上错误的解释。现有幻觉基准测试很少测试这种失败模式，特别是在非西方语境和英语之外。我们引入M$^2$CQA，一个基于17个中东国家图像的多模态基准测试，配对英语、阿拉伯语及其方言的对比真实和反事实陈述。为了隔离超出原始准确率的幻觉，我们提出反事实幻觉率（CFHR），衡量在正确回答真实陈述条件下接受反事实的条件。评估最先进的VLMs在多种提示策略下，发现阿拉伯语，尤其是方言中CFHR显著上升，即使真实陈述准确率保持高。此外，推理优先提示一致增加反事实幻觉，而先回答后解释则提高鲁棒性。我们公开发布该数据集供社区使用（https://huggingface.co/datasets/QCRI/M2CQA）。
英文摘要：
Vision-language models (VLMs) can achieve high accuracy while still accepting culturally plausible but visually incorrect interpretations. Existing hallucination benchmarks rarely test this failure mode, particularly outside Western contexts and English. We introduce M$^2$CQA, a culturally grounded multimodal benchmark built from images spanning 17 MENA countries, paired with contrastive true and counterfactual statements in English, Arabic, and its dialects. To isolate hallucination beyond raw accuracy, we propose the CounterFactual Hallucination Rate (CFHR), which measures counterfactual acceptance conditioned on correctly answering the true statement. Evaluating state-of-the-art VLMs under multiple prompting strategies, we find that CFHR rises sharply in Arabic, especially in dialects, even when true-statement accuracy remains high. Moreover, reasoning-first prompting consistently increases counterfactual hallucination, while answering before justifying improves robustness. We make the dataset publicly available for the community (https://huggingface.co/datasets/QCRI/M2CQA)).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
