arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.29428cs.CV

看到证据,却错失答案:基于工具引导的视觉语言模型在视觉错觉中的应用

Seeing the Evidence, Missing the Answer: Tool-Guided Vision-Language Models on Visual Illusions

Wayne State University(韦恩州立大学);University of Michigan(密歇根大学)

Xuesong Wang, Harry Wang

AI总结:

本文提出一种工具引导的推理框架,通过通用图像处理工具和提示系统,解决视觉语言模型在处理视觉错觉时的系统性偏差问题,并展示其在不同结构错觉变体上的跨结构泛化能力。

摘要:

视觉语言模型(VLMs)在面对经典光学错觉时表现出系统性偏差:无论图像是否被反事实修改,它们都倾向于预测错觉为'真实'。我们为DataCV 2026挑战(任务I和II)提出了一种工具引导的推理框架,该框架无需任何模型训练即可解决这一失败模式。一个现成的视觉语言模型被赋予一组通用图像处理工具:线描、区域裁剪、并列比较和通道隔离,以及一个根据感知问题类别指定调用哪些工具的错觉类型路由提示系统。关键的是,每次工具调用都会生成一个新的、不可变的图像资源,并附加到持久注册表中,因此模型可以在其推理链中参考和组合任何先前注释的视图。与硬编码错觉特定模块不同,这种通用工具加路由设计实现了强大的跨结构泛化:从验证集到包含结构不熟悉的错觉变体的测试集,性能保持一致。我们还报告了三个经验观察结果,我们认为值得进一步研究:(i) 可能源于错觉训练数据不平衡的强正检测偏差,(ii) 像素精确的空间推理与基于自动生成注释的逻辑推理之间的显著脱节,以及(iii) 对图像压缩伪影的明显敏感性,这会加剧假阳性。

英文摘要:

Vision-language models (VLMs) exhibit a systematic bias when confronted with classic optical illusions: they overwhelmingly predict the illusion as "real" regardless of whether the image has been counterfactually modified. We present a tool-guided inference framework for the DataCV 2026 Challenge (Tasks I and II) that addresses this failure mode without any model training. An off-the-shelf vision-language model is given access to a small set of generic image manipulation tools: line drawing, region cropping, side-by-side comparison, and channel isolation, together with an illusion-type-routing system prompt that prescribes which tools to invoke for each perceptual question category. Critically, every tool call produces a new, immutable image resource appended to a persistent registry, so the model can reference and compose any prior annotated view throughout its reasoning chain. Rather than hard-coding illusion-specific modules, this generic-tool-plus-routing design yields strong cross-structural generalization: performance remained consistent from the validation set to a test set containing structurally unfamiliar illusion variants (e.g., Mach Bands rotated from vertical to horizontal stacking). We further report three empirical observations that we believe warrant additional investigation: (i) a strong positive-detection bias likely rooted in imbalanced illusion training data, (ii) a striking dissociation between pixel-accurate spatial reasoning and logical inference over self-generated annotations, and (iii) pronounced sensitivity to image compression artifacts that compounds false positives.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑