视觉语言模型是感知还是记忆？通过经典视觉错觉探测视觉感知与记忆
Do VLMs Perceive or Recall? Probing Visual Perception vs. Memory with Classic Visual Illusions
AI总结：
本文通过经典视觉错觉研究视觉语言模型的感知与记忆机制，提出VI-Probe框架，揭示不同模型对视觉变化的响应来源，挑战单一原因观点。
摘要：
大型视觉-语言模型（VLMs）在原始图像上通常能正确回答经典视觉错觉，但当错觉因素反转时仍保持相同回答，尽管人类能明显察觉视觉变化。这引发了一个根本问题：VLMs是感知视觉变化还是仅回忆记忆中的模式？尽管已有研究注意到这一现象，但其根本原因仍不明确。为从观察走向系统理解，本文引入VI-Probe，一个可控的视觉错觉框架，具有渐进扰动和匹配视觉对照（无错觉诱导器），以分离视觉基础的感知与语言驱动的记忆。不同于以往关注平均准确率的研究，我们使用极性翻转一致性、模板注视指数和归一化后的错觉倍数来测量稳定性与敏感性。在不同家族的实验中发现，响应持续性源于异质原因而非单一机制。例如，GPT-5表现出记忆覆盖，Claude-Opus-4.1显示感知-记忆竞争，而Qwen变体则表明视觉处理限制。我们的发现挑战了单一原因观点，并推动基于探测的评估，测量知识和对受控视觉变化的敏感性。数据和代码可在https://sites.google.com/view/vi-probe/获取。
英文摘要：
Large Vision-Language Models (VLMs) often answer classic visual illusions "correctly" on original images, yet persist with the same responses when illusion factors are inverted, even though the visual change is obvious to humans. This raises a fundamental question: do VLMs perceive visual changes or merely recall memorized patterns? While several studies have noted this phenomenon, the underlying causes remain unclear. To move from observations to systematic understanding, this paper introduces VI-Probe, a controllable visual-illusion framework with graded perturbations and matched visual controls (without illusion inducer) that disentangles visually grounded perception from language-driven recall. Unlike prior work that focuses on averaged accuracy, we measure stability and sensitivity using Polarity-Flip Consistency, Template Fixation Index, and an illusion multiplier normalized against matched controls. Experiments across different families reveal that response persistence arises from heterogeneous causes rather than a single mechanism. For instance, GPT-5 exhibits memory override, Claude-Opus-4.1 shows perception-memory competition, while Qwen variants suggest visual-processing limits. Our findings challenge single-cause views and motivate probing-based evaluation that measures both knowledge and sensitivity to controlled visual change. Data and code are available at https://sites.google.com/view/vi-probe/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
