视觉语言模型通过视觉投影器中的语义纠缠将标识映射为文本
Vision Language Models Map Logos to Text via Semantic Entanglement in the Visual Projector
AI总结：
本研究揭示视觉语言模型在标识幻觉中依赖符号先验而非字形感知，通过系统测量和嵌入分析定位投影器子空间为关键因素，并提出解缠与OCR引导解码的缓解方向。
摘要：
视觉语言模型（VLMs）在多模态推理方面取得了显著进展；然而，它们仍然容易产生幻觉，即输出未基于视觉证据。本文研究了一个此前被忽视的场景：标识幻觉，即模型生成品牌名称或文本内容，尽管标识中不包含可见文字。利用精心划分的纯符号、混合型及含文本标识数据集，以及具有挑战性的Hard-60子集，我们系统地测量了主流视觉语言模型中的幻觉现象。我们进一步通过九种结构化扰动探测鲁棒性，并表明即使在强失真下幻觉仍然存在，其中遮挡暴露了最严重的弱点。使用开放权重的LLaVA进行嵌入级分析表明，幻觉与投影器维度的一小部分子集相关，针对性的消融在保持OCR准确性的同时大幅减少了错误。综合来看，这些发现揭示了视觉语言模型往往依赖符号先验而非真正的字形感知，尤其是对于标志性的圆形标识，并且投影器子空间在这一失败模式中起着决定性作用。我们的工作既提供了新颖的诊断视角，也提供了可行的缓解见解，强调了投影器解缠和OCR引导解码作为构建更可信多模态系统的有前景方向。
英文摘要：
Vision Language Models (VLMs) have achieved impressive progress in multimodal reasoning; yet, they remain vulnerable to hallucinations, where outputs are not grounded in visual evidence. In this paper, we investigate a previously overlooked setting: logo hallucination, where models generate brand names or textual content despite logos containing no visible words. Using curated splits of pure symbols, hybrids, and text-bearing logos, as well as the challenging Hard-60 subset, we systematically measure hallucination across leading VLMs. We further probe robustness through nine structured perturbations and show that hallucinations persist even under strong distortions, with occlusion exposing the sharpest weaknesses. Embedding-level analysis with open-weight LLaVA demonstrates that hallucination is tied to a small subset of projector dimensions, and targeted ablation substantially reduces errors while preserving OCR accuracy. Together, these findings reveal that VLMs often rely on symbolic priors rather than genuine glyph perception, particularly for iconic circular logos, and that projector subspaces play a decisive role in this failure mode. Our work contributes both a novel diagnostic lens and actionable mitigation insights, highlighting projector disentanglement and OCR-guided decoding as promising directions for building more trustworthy multimodal systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
