空间色彩混合错觉作为视觉-语言模型的感知压力测试
Spatial Colour Mixing Illusions as a Perception Stress Test for Vision-Language Models
AI总结：
研究通过空间色彩混合错觉测试视觉-语言模型的感知鲁棒性，发现模型在结构化色彩扭曲下表现下降，而人类表现更优，预处理步骤可提升模型性能。
摘要：
视觉-语言模型（VLMs）在基准测试中表现强劲，但仍然表现出系统性的感知弱点：对像素值的结构化、大幅更改会导致模型自信但不合逻辑的预测，即使底层场景仍能被人类轻易识别。我们通过空间色彩混合，一种程序化的色彩扭曲家族，将结构化图案（在RGB和奥斯特瓦尔德色彩系统中）叠加到自然图像上，来研究这一差距。我们引入了八个空间色彩混合变体的框架，并在三个模型家族的九个数据集上评估了VLMs。在模型和数据集上，准确性随着扭曲程度的增加而急剧下降，而扩大语言模型的规模并不能可靠地缓解失败。在一项包含61名参与者的动物识别数据集的人类研究中，人类在相同的扭曲下显著优于VLMs。最后，我们展示了一种简单的受人类启发的预处理步骤可以恢复几种扭曲类型的有意义性能部分，这激励了感知意识的预处理和工具使用作为改进VLM鲁棒性的实用策略。
英文摘要：
Vision-language models (VLMs) achieve strong benchmark results, yet can exhibit systematic perceptual weaknesses: structured, large changes to pixel values can cause confident yet nonsensical predictions, even when the underlying scene remains easily recognizable to humans. We study this gap using Spatial Colour Mixing, a programmatic family of colour distortions that overlays structured patterns (in both RGB and Ostwald colour systems) onto natural images. We introduce a framework of eight spatial colour mixing variants and evaluate nine VLMs across three model families on four datasets. Across models and datasets, accuracy degrades sharply with increasing distortion, and scaling the language model does not reliably mitigate the failure. In a human study with 61 participants on an animal recognition dataset, humans substantially outperform VLMs under the same distortions. Finally, we show that a simple human-inspired preprocessing step recovers a meaningful portion of performance for several distortion types, motivating perception-aware preprocessing and tool-use as practical strategies for improving VLM robustness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
