文本是什么颜色？：一个用于评估图像嵌入提示引发幻觉的基准
What Color Is the Text? A Benchmark for Hallucination Induced by Image-Embedded Prompt
AI总结：
该研究提出Embedded Stroop范式，构建WCIT基准评估MLLMs的图像嵌入提示幻觉，发现语义可读性可主导其视觉颜色感知。
摘要：
我们提出了Embedded Stroop，这是一种用于测量多模态大语言模型（MLLMs）中图像嵌入提示干扰的受控诊断范式，其中查询直接在视觉输入中呈现。使用What-Color-Is-the-Text（WCIT）基准（涵盖59种细粒度颜色，包含标准、翻转和掩码三种变体），我们评估了16种专有和开源模型。为了区分语义捕获与一般颜色命名失败，我们将模型响应分解为准确率、Stroop幻觉率（SHR；回答嵌入的词语而非真实文本颜色）和其他错误率，并通过置换检验验证该效应，在FDR校正后64个条件中有56个仍保持显著。尽管在59色词汇下精确颜色准确率较低（6.3%），但将预测映射到11个基本颜色族后显示，模型保留了粗略的颜色感知（38.4%），同时仍表现出显著的SHR（21.6%）。对标准设置下正确命名颜色的条件分析进一步证实了该效应，合并后的条件SHR超过50%。掩码或翻转嵌入文本可减少Stroop幻觉，表明在MLLMs中语义可读性可主导视觉颜色感知。
英文摘要：
We introduce Embedded Stroop, a controlled diagnostic paradigm for measuring image-embedded prompt interference in Multimodal Large Language Models (MLLMs), where the query is rendered directly inside the visual input. Using the What-Color-Is-the-Text (WCIT) benchmark, which covers 59 fine-grained colors under Standard, Flipped, and Masked variants, we evaluate 16 proprietary and open-source models. To distinguish semantic capture from general color-naming failure, we decompose model responses into Accuracy, Stroop Hallucination Rate (SHR; answering the embedded word rather than the true text color), and Other Error Rate, and validate the effect with permutation tests, with 56 of 64 conditions remaining significant after FDR correction. Although exact color accuracy is low (6.3%) under the 59-color vocabulary, mapping predictions to 11 basic color families shows that models retain coarse color perception (38.4%) while still exhibiting a substantial SHR (21.6%). A conditional analysis restricted to colors correctly named in the Standard setting further confirms the effect, with pooled conditional SHR exceeding 50\%. Masking or flipping the embedded text reduces Stroop hallucinations, suggesting that semantic legibility can dominate visual color perception in MLLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
