当视觉证据模糊时： pareidolia 作为视觉模型的诊断探针
When Visual Evidence is Ambiguous: Pareidolia as a Diagnostic Probe for Vision Models
AI总结：
本文研究了视觉模型在模糊证据下的行为，通过分析pareidolia现象揭示了不同模型在表示层面的差异，发现语义过激活和不确定性策略等机制，为提升视觉语言系统的语义鲁棒性提供诊断和改进方法。
摘要：
当视觉证据模糊时，视觉模型必须决定是否将面状模式解释为有意义的。面孔 pareidolia，即在非面孔物体中感知到面孔，提供了一个受控的探测此行为的手段。我们引入了一个表示层面的诊断框架，该框架分析了在面孔 pareidolia 图像中，跨类别、难度和情绪的检测、定位、不确定性和偏差。在统一的协议下，我们评估了六个模型，涵盖了四个表示领域：视觉语言模型（VLMs；CLIP-B/32，CLIP-L/14，LLaVA-1.5-7B）、纯视觉分类（ViT）、通用物体检测（YOLOv8）和人脸检测（RetinaFace）。我们的分析揭示了三种在模糊性下的解释机制。VLMs表现出语义过激活，系统地将模糊的非人类区域拉向人类概念，其中LLaVA-1.5-7B产生最强且最自信的过激活，尤其在负面情绪方面。ViT则遵循不确定性作为退避策略，保持模糊但总体偏见较小。基于检测的模型通过保守的先验知识实现低偏见，即使在定位受控的情况下，也能抑制pareidolia响应。这些结果表明，在模糊性下的行为更多由表示选择而非分数阈值决定，并且不确定性与偏见是解耦的：低不确定性可以表示安全的抑制，如在检测器中，或极端的过度解释，如在VLMs中。Pareidolia因此提供了一个紧凑的诊断手段以及一个模糊性感知的硬负样本来源，用于探测和改进视觉语言系统的语义鲁棒性。代码将在发表时发布。
英文摘要：
When visual evidence is ambiguous, vision models must decide how to interpret face-like patterns. Face pareidolia, the perception of faces in non-face objects, provides a controlled probe of such decisions. We introduce a diagnostic framework that analyzes detection, localization, uncertainty and bias across class, difficulty and emotion. We evaluate six models spanning four representational regimes: vision-language models (VLMs; CLIP-B/32, CLIP-L/14, LLaVA-1.5-7B), pure vision classification (ViT), object detection (YOLOv8), and face detection (RetinaFace). Our results reveal that uncertainty and bias are decoupled: low uncertainty can signal either safe suppression, as in detectors, or extreme over-interpretation, as in VLMs. VLMs exhibit semantic overactivation, systematically interpreting ambiguous non-human regions as Human, with LLaVA over-calling on 73% of non-human pareidolic images, especially for negative emotions. ViT instead follows an uncertainty-as-abstention strategy, remaining diffuse yet largely unbiased. Detection-based models achieve low bias through conservative priors that suppress pareidolia responses even when localization is controlled. Together, these results show that behavior under ambiguity is governed more by representation than thresholds, establishing face pareidolia as a diagnostic of semantic robustness and a source of ambiguity-aware hard negatives for vision models. Code will be released upon publication.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
