SMSP：多尺度感知的插件策略用于MLLMs感知视觉错觉
SMSP: A Plug-and-Play Strategy of Multi-Scale Perception for MLLMs to Perceive Visual Illusions
AI总结：
本文提出SMSP策略，通过抑制高频背景以提升MLLMs对视觉错觉的识别能力，实验表明其显著提高模型性能。
摘要：
近期研究表明，多模态大语言模型（MLLMs）对隐藏模式视觉错觉高度敏感，其隐藏内容对模型不可见但对人类明显。为系统研究此缺陷，本文引入IlluChar数据集，揭示模型失败的机制为高频注意力偏差，即模型易受图像中高频背景纹理干扰而忽略隐藏模式。为此，提出SMSP策略，一种插件式框架，与人类视觉感知策略对齐。通过抑制干扰的高频背景，SMSP生成更接近人类感知的图像。实验表明，SMSP显著提升所有评估的MLLMs在错觉图像上的性能，例如将Qwen3-VL-8B-Instruct的准确率从13.0%提升至84.0%。本文为MLLMs的视觉感知提供了新见解，并提供了一种实用且稳健的解决方案以增强其能力。代码已公开在https://github.com/Tujz2023/SMSP。
英文摘要：
Recent works have shown that multimodal large language models (MLLMs) are highly vulnerable to hidden-pattern visual illusions, where the hidden content is imperceptible to models but obvious to humans. This deficiency highlights a perceptual misalignment between current MLLMs and humans, and also introduces potential safety concerns. To systematically investigate this failure, we introduce IlluChar, a comprehensive and challenging illusion dataset, and uncover a key underlying mechanism for the models' failure: high-frequency attention bias, where the models are easily distracted by high-frequency background textures in illusion images, causing them to overlook hidden patterns. To address the issue, we propose the Strategy of Multi-Scale Perception (SMSP), a plug-and-play framework that aligns with human visual perceptual strategies. By suppressing distracting high-frequency background signals, SMSP generates images closer to human perception. Our experiments demonstrate that SMSP significantly improves the performance of all evaluated MLLMs on illusion images, for instance, increasing the accuracy of Qwen3-VL-8B-Instruct from 13.0% to 84.0%. Our work provides novel insights into MLLMs' visual perception, and offers a practical and robust solution to enhance it. Our code is publicly available at https://github.com/Tujz2023/SMSP.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
